Sep 06 05:54:06.179349 ip-10-0-129-33 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 05:54:06.583216 ip-10-0-129-33 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 05:54:06.583216 ip-10-0-129-33 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 05:54:06.583216 ip-10-0-129-33 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 05:54:06.583216 ip-10-0-129-33 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 05:54:06.583216 ip-10-0-129-33 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 05:54:06.585804 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.585628    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 05:54:06.590185 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590166    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.590185 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590184    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.590185 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590188    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590191    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590194    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590197    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590200    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590203    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590206    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590219    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590222    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590225    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590229    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590233    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590236    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590239    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590242    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590244    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590247    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590250    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590252    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590255    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.590281 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590257    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590260    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590263    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590265    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590268    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590271    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590274    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590276    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590278    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590281    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590283    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590286    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590289    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590292    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590295    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590297    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590301    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590304    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590307    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590309    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.590752 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590312    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590316    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590319    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590322    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590325    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590328    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590331    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590333    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590336    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590339    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590341    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590344    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590347    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590350    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590354    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590357    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590362    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590366    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590369    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.591256 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590372    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590375    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590377    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590380    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590383    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590385    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590388    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590391    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590394    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590396    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590399    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590402    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590405    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590407    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590410    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590412    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590415    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590417    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590420    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590423    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.591735 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590425    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590428    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590430    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590433    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590436    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590438    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590441    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590444    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590447    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590450    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590452    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590457    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590459    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590462    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590464    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590467    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590470    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.590472    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591198    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.592226 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591206    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591211    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591215    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591218    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591222    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591225    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591228    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591231    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591234    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591237    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591239    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591242    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591244    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591247    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591249    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591253    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591255    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591258    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591261    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.592689 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591264    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591269    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591271    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591274    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591277    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591280    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591283    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591285    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591288    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591291    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591293    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591296    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591299    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591301    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591304    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591306    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591309    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591312    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591315    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591318    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.593184 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591321    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591323    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591326    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591329    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591331    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591334    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591336    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591339    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591341    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591344    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591347    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591350    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591353    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591356    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591359    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591361    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591364    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591367    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591370    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591373    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.593674 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591376    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591378    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591381    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591383    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591386    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591389    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591391    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591394    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591397    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591401    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591403    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591407    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591410    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591413    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591416    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591418    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591421    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591423    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591426    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.594193 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591431    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591435    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591438    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591441    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591444    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591447    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591449    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591452    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591455    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591457    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591460    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591463    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591465    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591468    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591470    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591473    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591476    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591478    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591481    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591483    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.594663 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591913    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591918    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591922    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591925    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591929    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591932    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591935    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591938    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591941    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591944    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591947    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591950    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591953    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591956    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591958    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591962    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591965    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591967    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591970    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591974    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.595192 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591977    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591980    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591983    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591986    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591989    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591994    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591997    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.591999    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592002    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592004    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592007    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592009    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592013    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592016    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592019    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592022    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592025    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592028    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592031    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592033    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.595699 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592035    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592038    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592041    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592043    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592046    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592049    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592051    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592054    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592070    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592073    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592076    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592079    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592081    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592084    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592086    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592089    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592092    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592096    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592098    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.596200 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592103    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592107    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592110    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592112    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592115    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592117    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592120    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592123    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592125    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592128    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592130    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592134    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592136    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592139    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592141    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592144    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592147    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592149    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592151    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.596664 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592154    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592158    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592161    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592163    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592166    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592168    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592171    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592174    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592176    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592179    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592181    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592185    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592187    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592190    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592192    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592195    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592197    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592200    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592202    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592205    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.597179 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592207    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592275    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592310    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592316    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592321    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592326    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592329    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592333    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592338    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592341    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592345    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592348    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592351    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592355    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592359    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592362    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592365    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592368    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592370    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592375    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592377    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592381    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592384    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592387    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 05:54:06.597672 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592391    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592394    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592398    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592401    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592404    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592407    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592410    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592413    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592415    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592420    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592423    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592426    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592429    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592432    2567 flags.go:64] FLAG: --enable-server="true"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592436    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592441    2567 flags.go:64] FLAG: --event-burst="100"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592444    2567 flags.go:64] FLAG: --event-qps="50"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592447    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592450    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592452    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592456    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592459    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592462    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592465    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592468    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 05:54:06.598304 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592471    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592474    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592477    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592480    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592483    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592486    2567 flags.go:64] FLAG: --feature-gates=""
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592491    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592494    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592498    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592501    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592504    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592507    2567 flags.go:64] FLAG: --help="false"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592511    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592514    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592517    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592520    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592524    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592527    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592530    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592533    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592535    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592538    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592542    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592545    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 05:54:06.598984 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592548    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592551    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592553    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592557    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592560    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592563    2567 flags.go:64] FLAG: --lock-file=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592566    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592569    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592572    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592578    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592581    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592584    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592586    2567 flags.go:64] FLAG: --logging-format="text"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592589    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592593    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592596    2567 flags.go:64] FLAG: --manifest-url=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592599    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592603    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592606    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592610    2567 flags.go:64] FLAG: --max-pods="110"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592614    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592617    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592619    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592623    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592626    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 05:54:06.599600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592629    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592632    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592639    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592642    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592645    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592648    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592652    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592657    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592660    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592664    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592667    2567 flags.go:64] FLAG: --port="10250"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592670    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592673    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0887f9fd96dd48d71"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592676    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592679    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592682    2567 flags.go:64] FLAG: --register-node="true"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592685    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592689    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592692    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592695    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592698    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592702    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592706    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592709    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592712    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 05:54:06.600244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592715    2567 flags.go:64] FLAG: --runonce="false"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592718    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592721    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592724    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592727    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592730    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592733    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592736    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592739    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592742    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592745    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592748    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592751    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592754    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592759    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592762    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592773    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592778    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592781    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592785    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592788    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592791    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592793    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592796    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592800    2567 flags.go:64] FLAG: --v="2"
Sep 06 05:54:06.600852 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592804    2567 flags.go:64] FLAG: --version="false"
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592808    2567 flags.go:64] FLAG: --vmodule=""
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592812    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.592816    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592934    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592938    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592942    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592946    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592949    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592952    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592955    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592958    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592961    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592964    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592966    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592969    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592972    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592975    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592977    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.601478 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592979    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592982    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592985    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592988    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592990    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592996    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.592998    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593001    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593003    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593006    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593009    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593011    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593014    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593017    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593021    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593024    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593028    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593031    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593034    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.601980 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593037    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593040    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593042    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593044    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593047    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593050    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593052    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593055    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593070    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593073    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593075    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593078    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593081    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593083    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593086    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593089    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593091    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593095    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593099    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593101    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.602483 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593104    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593107    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593109    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593112    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593115    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593117    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593120    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593123    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593125    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593129    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593131    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593134    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593137    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593139    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593142    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593145    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593147    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593150    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593153    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593155    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.602975 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593158    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593161    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593163    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593166    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593168    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593171    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593174    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593176    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593179    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593182    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593186    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593189    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593193    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593197    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593200    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593202    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593206    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593208    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593211    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593213    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.603498 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593216    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593220    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593223    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593225    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.593228    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.593233    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.600712    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.600731    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.602298    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.602307    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602378    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602384    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602387    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602391    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602394    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.603993 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602396    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602399    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602401    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602404    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602407    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602409    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602412    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602421    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602423    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602426    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602428    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602431    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602433    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602436    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602439    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602442    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602445    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602448    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602450    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602453    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.604399 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602456    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602458    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602461    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602463    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602466    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602469    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602472    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602474    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602478    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602483    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602485    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602489    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602492    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602495    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602499    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602502    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602505    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602507    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602510    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.604898 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602512    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602521    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602523    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602527    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602529    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602532    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602535    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602538    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602540    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602543    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602546    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602548    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602551    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602553    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602556    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602558    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602562    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602566    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602569    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602571    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.605605 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602574    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602577    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602579    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602582    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602584    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602587    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602589    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602592    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602595    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602599    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602601    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602604    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602606    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602609    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602617    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602621    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602624    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602627    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602629    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.606113 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602632    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602634    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602643    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602645    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602648    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602650    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602653    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602656    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602658    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602661    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602664    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602667    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602669    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602672    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602674    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.606584 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602677    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.602682    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602842    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602847    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602850    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602853    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602855    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602858    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602861    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602863    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602866    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602869    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602871    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602881    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602884    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 05:54:06.606958 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602887    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602889    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602892    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602895    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602897    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602900    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602902    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602905    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602907    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602910    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602912    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602915    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602918    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602921    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602923    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602925    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602928    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602931    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602933    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 05:54:06.607354 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602936    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602938    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602941    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602943    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602945    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602948    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602950    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602953    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602955    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602958    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602960    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602963    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602971    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602974    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602977    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602979    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602982    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602985    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602987    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602990    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 05:54:06.607822 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602992    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602995    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.602998    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603000    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603002    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603005    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603007    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603010    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603012    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603015    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603018    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603020    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603023    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603025    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603027    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603030    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603032    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603035    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603037    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 05:54:06.608643 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603040    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603044    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603047    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603049    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603052    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603054    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603076    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603081    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603084    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603087    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603090    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603093    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603095    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603098    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603101    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603104    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603108    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603111    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603115    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 05:54:06.609297 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603118    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603121    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603124    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603126    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603129    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603131    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603134    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603136    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:06.603139    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.603144    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.603276    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.605226    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.606171    2567 server.go:1019] "Starting client certificate rotation"
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.606277    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 05:54:06.609819 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.606316    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 05:54:06.611500 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.611482    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 05:54:06.617324 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.617304    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 05:54:06.629527 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.629502    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 06 05:54:06.633101 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.633082    2567 log.go:25] "Validated CRI v1 image API"
Sep 06 05:54:06.634625 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.634610    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 05:54:06.637925 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.637904    2567 fs.go:135] Filesystem UUIDs: map[2b85f4ea-0e66-41b0-bcaf-235c29be3a23:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a98a49d4-c672-4fd4-94e4-deadb478f1d5:/dev/nvme0n1p3]
Sep 06 05:54:06.637996 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.637923    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 05:54:06.638279 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.638257    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 05:54:06.643698 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.643585    2567 manager.go:216] Machine: {Timestamp:2026-09-06 05:54:06.642151143 +0000 UTC m=+0.345703870 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103659 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a6d18b347341591970d6c152f1abf SystemUUID:ec2a6d18-b347-3415-9197-0d6c152f1abf BootID:c62d7ef8-473c-4233-9de5-668abc24f545 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6c:76:59:60:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6c:76:59:60:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:d2:79:b8:76:10 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 05:54:06.643698 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.643696    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 05:54:06.643811 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.643780    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 05:54:06.646446 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.646421    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 05:54:06.646587 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.646448    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 05:54:06.647658 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.647647    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 05:54:06.647699 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.647661    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 05:54:06.647699 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.647668    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 05:54:06.648498 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.648487    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 05:54:06.649680 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.649670    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 05:54:06.652150 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652137    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 05:54:06.652278 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652269    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 05:54:06.652326 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652318    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 05:54:06.652354 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652334    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 05:54:06.652387 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652357    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 05:54:06.652387 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652370    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 06 05:54:06.652387 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652380    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 05:54:06.652790 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652776    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 05:54:06.652825 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.652804    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 05:54:06.656437 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.656415    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 05:54:06.656809 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.656791    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdksm"
Sep 06 05:54:06.658685 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.658671    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 05:54:06.658724 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.658701    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 05:54:06.659563 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659539    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 05:54:06.659563 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659558    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 05:54:06.659563 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659564    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 05:54:06.659563 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659570    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659579    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659584    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659597    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659602    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659609    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659615    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659623    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 05:54:06.659745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.659628    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 05:54:06.660351 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.660340    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 05:54:06.660351 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.660351    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 05:54:06.664031 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.664012    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdksm"
Sep 06 05:54:06.665008 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.664986    2567 server.go:1268] "Started kubelet"
Sep 06 05:54:06.665132 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.665085    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 05:54:06.665231 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.665155    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 05:54:06.665510 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.665485    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 05:54:06.665582 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.665532    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 05:54:06.665876 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.665854    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 05:54:06.665876 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.665852    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 05:54:06.666000 ip-10-0-129-33 systemd[1]: Started Kubernetes Kubelet.
Sep 06 05:54:06.666303 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.666289    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 05:54:06.668048 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.668032    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 05:54:06.670191 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670175    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 05:54:06.670290 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670193    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 05:54:06.670334 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.670288    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:06.670899 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670782    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 05:54:06.670899 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670786    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 05:54:06.670899 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670808    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 05:54:06.671101 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670911    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 05:54:06.671101 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.670920    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 05:54:06.671101 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.670959    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 05:54:06.671223 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671162    2567 factory.go:55] Registering systemd factory
Sep 06 05:54:06.671223 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671177    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 06 05:54:06.671392 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671377    2567 factory.go:153] Registering CRI-O factory
Sep 06 05:54:06.671430 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671397    2567 factory.go:223] Registration of the crio container factory successfully
Sep 06 05:54:06.671472 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671449    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 05:54:06.671472 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671468    2567 factory.go:103] Registering Raw factory
Sep 06 05:54:06.671533 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671482    2567 manager.go:1196] Started watching for new ooms in manager
Sep 06 05:54:06.671891 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.671879    2567 manager.go:349] Starting recovery of all containers
Sep 06 05:54:06.672436 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.672418    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:06.676547 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.676518    2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-33.ec2.internal\" not found" node="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.683732 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.683693    2567 manager.go:354] Recovery completed
Sep 06 05:54:06.688687 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.688669    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.690818 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.690799    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.690909 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.690835    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.690909 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.690851    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.691525 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.691499    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 05:54:06.691525 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.691512    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 05:54:06.691655 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.691527    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.691655 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.691532    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 05:54:06.692707 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.692695    2567 policy_none.go:49] "None policy: Start"
Sep 06 05:54:06.692749 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.692714    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 05:54:06.692749 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.692727    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 05:54:06.693529 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.693513    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.693596 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.693547    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.693596 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.693558    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.693949 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.693939    2567 policy_none.go:47] "Start"
Sep 06 05:54:06.723790 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.723773    2567 manager.go:337] "Starting Device Plugin manager"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.723817    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.723829    2567 server.go:87] "Starting device plugin registration server"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.724101    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.724112    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.724208    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.724307    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.724315    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.725023    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 05:54:06.737237 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.725104    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:06.778777 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.778742    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 05:54:06.779949 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.779926    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 05:54:06.780050 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.780000    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 05:54:06.780050 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.780030    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 05:54:06.780174 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.780152    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 05:54:06.782213 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.782194    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:06.825117 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.825080    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.826396 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.826380    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.826487 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.826412    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.826487 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.826426    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.826487 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.826455    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.835521 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.835476    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.835521 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.835502    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-33.ec2.internal\": node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:06.850357 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.850329    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:06.880724 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.880563    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"]
Sep 06 05:54:06.880792 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.880787    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.882306 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.882290    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.882368 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.882320    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.882368 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.882334    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.883404 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.883393    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.883521 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.883508    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.883569 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.883535    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.884133 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884120    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.884202 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884144    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.884202 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884157    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.884599 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884586    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.884662 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884614    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.884662 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.884632    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.885200 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.885186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.885260 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.885217    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 05:54:06.885864 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.885848    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 05:54:06.885938 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.885876    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 05:54:06.885938 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.885886    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeHasSufficientPID"
Sep 06 05:54:06.906709 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.906689    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-33.ec2.internal\" not found" node="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.910976 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.910961    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-33.ec2.internal\" not found" node="ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.951033 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:06.951010    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:06.972188 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.972162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.972291 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.972190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:06.972291 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:06.972207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00d162c63e419062586c3987b0b951dc-config\") pod \"kube-apiserver-proxy-ip-10-0-129-33.ec2.internal\" (UID: \"00d162c63e419062586c3987b0b951dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.051949 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.051911    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.073279 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.073361 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.073361 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.073436 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00d162c63e419062586c3987b0b951dc-config\") pod \"kube-apiserver-proxy-ip-10-0-129-33.ec2.internal\" (UID: \"00d162c63e419062586c3987b0b951dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.073436 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073391    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecdb99928c19c8ca2cd1bded8a5a374-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal\" (UID: \"2ecdb99928c19c8ca2cd1bded8a5a374\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.073503 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.073438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00d162c63e419062586c3987b0b951dc-config\") pod \"kube-apiserver-proxy-ip-10-0-129-33.ec2.internal\" (UID: \"00d162c63e419062586c3987b0b951dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.152619 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.152536    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.209161 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.209134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.213736 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.213721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:07.253010 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.252980    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.353612 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.353578    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.456055 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.455934    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.558777 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.558741    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.606221 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.606189    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 05:54:07.606886 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.606338    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 05:54:07.606886 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.606352    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 05:54:07.658855 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.658815    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.667883 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.667853    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 05:49:07 +0000 UTC" deadline="2028-02-19 12:21:41.972748176 +0000 UTC"
Sep 06 05:54:07.667950 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.667883    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12750h27m34.304868058s"
Sep 06 05:54:07.759736 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.759700    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.812485 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.812447    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:07.862596 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.862555    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:07.897654 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:07.897616    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d162c63e419062586c3987b0b951dc.slice/crio-cf9231f198423aa0926b346a48a2060fc5fc5e7514fb70faac634f6fa1cff5ea WatchSource:0}: Error finding container cf9231f198423aa0926b346a48a2060fc5fc5e7514fb70faac634f6fa1cff5ea: Status 404 returned error can't find the container with id cf9231f198423aa0926b346a48a2060fc5fc5e7514fb70faac634f6fa1cff5ea
Sep 06 05:54:07.897887 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:07.897868    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecdb99928c19c8ca2cd1bded8a5a374.slice/crio-0bab3a011edc03f5cef5689744fb419252eca10c029e6ce213aec2dd36fb24d2 WatchSource:0}: Error finding container 0bab3a011edc03f5cef5689744fb419252eca10c029e6ce213aec2dd36fb24d2: Status 404 returned error can't find the container with id 0bab3a011edc03f5cef5689744fb419252eca10c029e6ce213aec2dd36fb24d2
Sep 06 05:54:07.902874 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:07.902857    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:54:07.963636 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:07.963585    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:08.064531 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.064458    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:08.164557 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.164515    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-33.ec2.internal\" not found"
Sep 06 05:54:08.212952 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.212926    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:08.271181 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.271154    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:08.283568 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.283543    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 05:54:08.284713 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.284697    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal"
Sep 06 05:54:08.292888 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.292866    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 05:54:08.654349 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.654315    2567 apiserver.go:52] "Watching apiserver"
Sep 06 05:54:08.664287 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.664258    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 05:54:08.664774 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.664741    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal","openshift-multus/multus-cmg6t","openshift-multus/network-metrics-daemon-pvwwb","openshift-network-operator/iptables-alerter-tfp5n","kube-system/global-pull-secret-syncer-wsczd","kube-system/konnectivity-agent-8kdpv","openshift-cluster-node-tuning-operator/tuned-zwl4k","openshift-dns/node-resolver-sq2fh","openshift-image-registry/node-ca-fdf8l","openshift-multus/multus-additional-cni-plugins-j4ktq","openshift-network-diagnostics/network-check-target-nk5nq","openshift-ovn-kubernetes/ovnkube-node-cx2k4","kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"]
Sep 06 05:54:08.667666 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.667643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.671393    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.671653    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.671795    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wtxjc\""
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.671920    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.671992    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:08.672364 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.672058    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:08.674174 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.673819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.674174 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.673888    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:08.676226 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.676178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.678875 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.678849    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.678961 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.678917    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:08.679190 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.679159    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.679299 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.679238    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 05:54:08.679381 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.679338    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.679509 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.679488    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v9d7h\""
Sep 06 05:54:08.680982 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.680960    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.682924 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.682904    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 05:54:08.683176 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-sys\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683272 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-tmp\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683272 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:08.683272 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683245    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-run\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-var-lib-kubelet\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd7tr\" (UniqueName: \"kubernetes.io/projected/e6aa072d-833e-4dbf-b55f-6e7b6951e690-kube-api-access-cd7tr\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-modprobe-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-kubernetes\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-lib-modules\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683419 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-host\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-tuned\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysconfig\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683483    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683516    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-systemd\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-conf\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.683711 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683650    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2z66f\""
Sep 06 05:54:08.684055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683941    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 05:54:08.684109 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.683941    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 05:54:08.686438 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.685989    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.686849 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.686726    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.688570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.688172    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.688570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.688236    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k5xwn\""
Sep 06 05:54:08.688570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.688456    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 05:54:08.689962 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.689256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.689962 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.689294    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.689962 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.689889    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9qfq8\""
Sep 06 05:54:08.690422 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.690399    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.690509 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.690425    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 05:54:08.692280 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.692159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.692889 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.692570    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.692889 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.692596    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 05:54:08.692889 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.692705    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.693194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.692960    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bjcz8\""
Sep 06 05:54:08.693194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.693127    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 05:54:08.694795 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.694776    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 05:54:08.695282 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.695025    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.695458 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.695163    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sqcw2\""
Sep 06 05:54:08.695548 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.695188    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 05:54:08.697613 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.697026    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.699968 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.697993    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z45b4\""
Sep 06 05:54:08.699968 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.698524    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.699968 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.698735    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.700798 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.700506    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r2fj7\""
Sep 06 05:54:08.701639 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.701617    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 05:54:08.702167 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.702130    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 05:54:08.702414 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.702396    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 05:54:08.702646 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.702629    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 05:54:08.702814 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.702797    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 05:54:08.702889 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.702879    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 05:54:08.705469 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.705446    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5t2dg"
Sep 06 05:54:08.712509 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.712488    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5t2dg"
Sep 06 05:54:08.771864 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.771839    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 05:54:08.783772 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysconfig\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.783880 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-registration-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.783880 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovn-node-metrics-cert\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.783880 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783846    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-socket-dir-parent\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysconfig\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cnibin\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz9g6\" (UniqueName: \"kubernetes.io/projected/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-kube-api-access-tz9g6\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.783993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-netns\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.784055 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-multus\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784126    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c060475-db7c-41ec-ac4b-5840c313ef83-tmp-dir\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-systemd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6swg\" (UniqueName: \"kubernetes.io/projected/03024c88-a2e9-4f74-9e27-c48e8cb493f6-kube-api-access-j6swg\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-lib-modules\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-host\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-sys-fs\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-hostroot\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784369    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-lib-modules\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phgsn\" (UniqueName: \"kubernetes.io/projected/0c060475-db7c-41ec-ac4b-5840c313ef83-kube-api-access-phgsn\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-host\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.784483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-system-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cnibin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07e0569e-4ba8-44c8-9dd3-10a7823e0848-iptables-alerter-script\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-tmp\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cni-binary-copy\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-conf-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46rb2\" (UniqueName: \"kubernetes.io/projected/fa8e1de9-0276-49d2-ad3c-4c4e79826925-kube-api-access-46rb2\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784736    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-dbus\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-konnectivity-ca\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd7tr\" (UniqueName: \"kubernetes.io/projected/e6aa072d-833e-4dbf-b55f-6e7b6951e690-kube-api-access-cd7tr\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784832    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-socket-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-device-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.784882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-systemd-units\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-var-lib-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.785258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-etc-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785134    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-ovn\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785166    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-config\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8ee0da7-abb2-4bcd-b504-6487a14049d6-serviceca\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-etc-kubernetes\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkp7\" (UniqueName: \"kubernetes.io/projected/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-kube-api-access-hqkp7\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785291    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rcvg\" (UniqueName: \"kubernetes.io/projected/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kube-api-access-4rcvg\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-k8s-cni-cncf-io\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhp8b\" (UniqueName: \"kubernetes.io/projected/07e0569e-4ba8-44c8-9dd3-10a7823e0848-kube-api-access-mhp8b\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-kubelet\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785440    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8ee0da7-abb2-4bcd-b504-6487a14049d6-host\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-netns\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-system-cni-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c060475-db7c-41ec-ac4b-5840c313ef83-hosts-file\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-kubelet-config\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.786199 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-slash\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-var-lib-kubelet\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-multus-certs\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785770    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-modprobe-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785795    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785811    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-var-lib-kubelet\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-modprobe-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.785838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07e0569e-4ba8-44c8-9dd3-10a7823e0848-host-slash\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-log-socket\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-netd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-systemd\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-conf\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-sys\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.786913 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786213    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626w2\" (UniqueName: \"kubernetes.io/projected/d8ee0da7-abb2-4bcd-b504-6487a14049d6-kube-api-access-626w2\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-daemon-config\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786258    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-conf\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-run\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-bin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-kubelet\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-systemd\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-agent-certs\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786383    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-sys\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-run\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-env-overrides\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-kubernetes\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-sysctl-d\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-tuned\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.787770 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-node-log\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.795575 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.795549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-bin\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.795686 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.795599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-script-lib\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.795825 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.786613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-kubernetes\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.797615 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.797591    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:08.797722 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.797619    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:08.797722 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.797634    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:08.797834 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.797791    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:09.297750372 +0000 UTC m=+3.001303111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:08.799206 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.799128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal" event={"ID":"2ecdb99928c19c8ca2cd1bded8a5a374","Type":"ContainerStarted","Data":"0bab3a011edc03f5cef5689744fb419252eca10c029e6ce213aec2dd36fb24d2"}
Sep 06 05:54:08.803319 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.803266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal" event={"ID":"00d162c63e419062586c3987b0b951dc","Type":"ContainerStarted","Data":"cf9231f198423aa0926b346a48a2060fc5fc5e7514fb70faac634f6fa1cff5ea"}
Sep 06 05:54:08.807164 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.807139    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-tmp\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.809098 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.809074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd7tr\" (UniqueName: \"kubernetes.io/projected/e6aa072d-833e-4dbf-b55f-6e7b6951e690-kube-api-access-cd7tr\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.810531 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.810482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6aa072d-833e-4dbf-b55f-6e7b6951e690-etc-tuned\") pod \"tuned-zwl4k\" (UID: \"e6aa072d-833e-4dbf-b55f-6e7b6951e690\") " pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.895872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-registration-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.895918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovn-node-metrics-cert\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.895944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.895968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-socket-dir-parent\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.895994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cnibin\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz9g6\" (UniqueName: \"kubernetes.io/projected/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-kube-api-access-tz9g6\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896044    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-netns\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-multus\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c060475-db7c-41ec-ac4b-5840c313ef83-tmp-dir\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-systemd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6swg\" (UniqueName: \"kubernetes.io/projected/03024c88-a2e9-4f74-9e27-c48e8cb493f6-kube-api-access-j6swg\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-sys-fs\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-socket-dir-parent\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-hostroot\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.897104 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-registration-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phgsn\" (UniqueName: \"kubernetes.io/projected/0c060475-db7c-41ec-ac4b-5840c313ef83-kube-api-access-phgsn\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896428    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-system-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cnibin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07e0569e-4ba8-44c8-9dd3-10a7823e0848-iptables-alerter-script\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896543    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cni-binary-copy\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-conf-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46rb2\" (UniqueName: \"kubernetes.io/projected/fa8e1de9-0276-49d2-ad3c-4c4e79826925-kube-api-access-46rb2\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-dbus\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-konnectivity-ca\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-socket-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-device-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-systemd-units\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-var-lib-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896801    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cnibin\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.898160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-etc-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-ovn\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-config\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8ee0da7-abb2-4bcd-b504-6487a14049d6-serviceca\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-etc-kubernetes\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkp7\" (UniqueName: \"kubernetes.io/projected/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-kube-api-access-hqkp7\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.896991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rcvg\" (UniqueName: \"kubernetes.io/projected/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kube-api-access-4rcvg\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-k8s-cni-cncf-io\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-netns\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-multus\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897283    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-sys-fs\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897295    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-hostroot\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897346    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-systemd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c060475-db7c-41ec-ac4b-5840c313ef83-tmp-dir\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.899115 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-device-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897934    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897993    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-system-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.898047    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cnibin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.898165    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.898492    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.898614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07e0569e-4ba8-44c8-9dd3-10a7823e0848-iptables-alerter-script\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.898701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-dbus\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-cni-binary-copy\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899099    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-conf-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899251    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-konnectivity-ca\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bcb82f-59d2-4f9a-8ac4-27201d22a233-socket-dir\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899447    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-etc-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-systemd-units\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovn-node-metrics-cert\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-var-lib-openvswitch\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8ee0da7-abb2-4bcd-b504-6487a14049d6-serviceca\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-etc-kubernetes\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.900110 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-run-ovn\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.901031 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.899844    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-k8s-cni-cncf-io\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.901031 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.900224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-config\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.901031 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.900354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.901194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.897078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhp8b\" (UniqueName: \"kubernetes.io/projected/07e0569e-4ba8-44c8-9dd3-10a7823e0848-kube-api-access-mhp8b\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.901194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-kubelet\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.901342 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901210    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8ee0da7-abb2-4bcd-b504-6487a14049d6-host\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.901342 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-netns\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.901342 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-system-cni-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.901342 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c060475-db7c-41ec-ac4b-5840c313ef83-hosts-file\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.901342 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-kubelet-config\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-slash\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-multus-certs\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07e0569e-4ba8-44c8-9dd3-10a7823e0848-host-slash\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-log-socket\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-netd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626w2\" (UniqueName: \"kubernetes.io/projected/d8ee0da7-abb2-4bcd-b504-6487a14049d6-kube-api-access-626w2\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-daemon-config\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-bin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-kubelet\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-agent-certs\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901725    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-multus-certs\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.903716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901778    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-kubelet\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-env-overrides\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-node-log\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901831    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-run-netns\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-bin\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-script-lib\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8e1de9-0276-49d2-ad3c-4c4e79826925-system-cni-dir\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901920    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c060475-db7c-41ec-ac4b-5840c313ef83-hosts-file\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eeb7275-b0a1-4dca-bc64-979136db13eb-kubelet-config\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.901993    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-cni-dir\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.902065    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-log-socket\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902116    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-netd\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.902154    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:09.402118576 +0000 UTC m=+3.105671310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902195    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-slash\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.902420    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:08.904786 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:08.902464    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:09.402449979 +0000 UTC m=+3.106002709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07e0569e-4ba8-44c8-9dd3-10a7823e0848-host-slash\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-multus-daemon-config\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-cni-bin\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902692    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8ee0da7-abb2-4bcd-b504-6487a14049d6-host\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.902741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-host-var-lib-kubelet\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-env-overrides\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903189    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-node-log\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903223    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8e1de9-0276-49d2-ad3c-4c4e79826925-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903243    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-cni-bin\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03024c88-a2e9-4f74-9e27-c48e8cb493f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.905660 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.903719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03024c88-a2e9-4f74-9e27-c48e8cb493f6-ovnkube-script-lib\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.908820 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.908797    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23f16e8-d43a-4b61-bb80-b7e0c39d5c25-agent-certs\") pod \"konnectivity-agent-8kdpv\" (UID: \"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25\") " pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:08.910054 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.910004    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46rb2\" (UniqueName: \"kubernetes.io/projected/fa8e1de9-0276-49d2-ad3c-4c4e79826925-kube-api-access-46rb2\") pod \"multus-additional-cni-plugins-j4ktq\" (UID: \"fa8e1de9-0276-49d2-ad3c-4c4e79826925\") " pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:08.910503 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.910479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz9g6\" (UniqueName: \"kubernetes.io/projected/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-kube-api-access-tz9g6\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:08.911025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.910950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phgsn\" (UniqueName: \"kubernetes.io/projected/0c060475-db7c-41ec-ac4b-5840c313ef83-kube-api-access-phgsn\") pod \"node-resolver-sq2fh\" (UID: \"0c060475-db7c-41ec-ac4b-5840c313ef83\") " pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:08.915480 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.915455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-626w2\" (UniqueName: \"kubernetes.io/projected/d8ee0da7-abb2-4bcd-b504-6487a14049d6-kube-api-access-626w2\") pod \"node-ca-fdf8l\" (UID: \"d8ee0da7-abb2-4bcd-b504-6487a14049d6\") " pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:08.916530 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.916114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhp8b\" (UniqueName: \"kubernetes.io/projected/07e0569e-4ba8-44c8-9dd3-10a7823e0848-kube-api-access-mhp8b\") pod \"iptables-alerter-tfp5n\" (UID: \"07e0569e-4ba8-44c8-9dd3-10a7823e0848\") " pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:08.916530 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.916160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkp7\" (UniqueName: \"kubernetes.io/projected/98c6cf0e-08b2-4db0-9e50-6ebfec68eff3-kube-api-access-hqkp7\") pod \"multus-cmg6t\" (UID: \"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3\") " pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:08.916706 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.916622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6swg\" (UniqueName: \"kubernetes.io/projected/03024c88-a2e9-4f74-9e27-c48e8cb493f6-kube-api-access-j6swg\") pod \"ovnkube-node-cx2k4\" (UID: \"03024c88-a2e9-4f74-9e27-c48e8cb493f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:08.918748 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.918278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rcvg\" (UniqueName: \"kubernetes.io/projected/98bcb82f-59d2-4f9a-8ac4-27201d22a233-kube-api-access-4rcvg\") pod \"aws-ebs-csi-driver-node-tdsv4\" (UID: \"98bcb82f-59d2-4f9a-8ac4-27201d22a233\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:08.939258 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.939234    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:08.952028 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.952005    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 05:54:08.982911 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.982881    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwl4k"
Sep 06 05:54:08.997744 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:08.997722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfp5n"
Sep 06 05:54:09.012722 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.012699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:09.020526 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.020507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdf8l"
Sep 06 05:54:09.029574 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.029554    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4"
Sep 06 05:54:09.041339 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.041322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cmg6t"
Sep 06 05:54:09.048164 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.048127    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4ktq"
Sep 06 05:54:09.057043 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.057023    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sq2fh"
Sep 06 05:54:09.064944 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.064924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:09.305025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.304952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:09.305200 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.305125    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:09.305200 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.305149    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:09.305200 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.305159    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:09.305308 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.305215    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:10.305197829 +0000 UTC m=+4.008750586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:09.405508 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.405466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:09.405717 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.405518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:09.405717 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.405638    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:09.405717 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.405648    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:09.405717 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.405719    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:10.405697016 +0000 UTC m=+4.109249736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:09.405879 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.405741    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:10.405730694 +0000 UTC m=+4.109283417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:09.716038 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.715941    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 05:49:09 +0000 UTC" deadline="2028-03-25 15:19:44.121566951 +0000 UTC"
Sep 06 05:54:09.716038 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.715978    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13593h25m34.405594264s"
Sep 06 05:54:09.780742 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:09.780704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:09.780897 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:09.780846    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:09.930928 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.930818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6aa072d_833e_4dbf_b55f_6e7b6951e690.slice/crio-64cd4ff6dbeee2ce912b3cb7b327c0fe3878819db2893296255f5f6b90bd32d4 WatchSource:0}: Error finding container 64cd4ff6dbeee2ce912b3cb7b327c0fe3878819db2893296255f5f6b90bd32d4: Status 404 returned error can't find the container with id 64cd4ff6dbeee2ce912b3cb7b327c0fe3878819db2893296255f5f6b90bd32d4
Sep 06 05:54:09.953556 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.953444    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c060475_db7c_41ec_ac4b_5840c313ef83.slice/crio-1c09f27ada2f8b7420bb137e1ae9459b7af6c4da52b9d662a8f9fbcb237c81d6 WatchSource:0}: Error finding container 1c09f27ada2f8b7420bb137e1ae9459b7af6c4da52b9d662a8f9fbcb237c81d6: Status 404 returned error can't find the container with id 1c09f27ada2f8b7420bb137e1ae9459b7af6c4da52b9d662a8f9fbcb237c81d6
Sep 06 05:54:09.956355 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.956331    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ee0da7_abb2_4bcd_b504_6487a14049d6.slice/crio-97aeb6a83ab2355fb950da7c2c3dcb6eb067fb63e55bc96f4084e3f4116fdbca WatchSource:0}: Error finding container 97aeb6a83ab2355fb950da7c2c3dcb6eb067fb63e55bc96f4084e3f4116fdbca: Status 404 returned error can't find the container with id 97aeb6a83ab2355fb950da7c2c3dcb6eb067fb63e55bc96f4084e3f4116fdbca
Sep 06 05:54:09.960744 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.960720    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03024c88_a2e9_4f74_9e27_c48e8cb493f6.slice/crio-41fb3fa0547dfa67d937cb9b3f96f3d600d366af8793c795234967027034c622 WatchSource:0}: Error finding container 41fb3fa0547dfa67d937cb9b3f96f3d600d366af8793c795234967027034c622: Status 404 returned error can't find the container with id 41fb3fa0547dfa67d937cb9b3f96f3d600d366af8793c795234967027034c622
Sep 06 05:54:09.962188 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.962166    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bcb82f_59d2_4f9a_8ac4_27201d22a233.slice/crio-6a4355a09eeee2e0bd054581b6c7306ccfb68a58371e7ba698f1032b020cf23f WatchSource:0}: Error finding container 6a4355a09eeee2e0bd054581b6c7306ccfb68a58371e7ba698f1032b020cf23f: Status 404 returned error can't find the container with id 6a4355a09eeee2e0bd054581b6c7306ccfb68a58371e7ba698f1032b020cf23f
Sep 06 05:54:09.963760 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.963399    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23f16e8_d43a_4b61_bb80_b7e0c39d5c25.slice/crio-60c2bea834e8d94bea422d4b2da288df4a2b9ae198750aac9e83498a90e75097 WatchSource:0}: Error finding container 60c2bea834e8d94bea422d4b2da288df4a2b9ae198750aac9e83498a90e75097: Status 404 returned error can't find the container with id 60c2bea834e8d94bea422d4b2da288df4a2b9ae198750aac9e83498a90e75097
Sep 06 05:54:09.967464 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.967243    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8e1de9_0276_49d2_ad3c_4c4e79826925.slice/crio-ef84e1d63f578ee8bb0a56a4dde6c28e1390afd288db52526f31b7f9a1253878 WatchSource:0}: Error finding container ef84e1d63f578ee8bb0a56a4dde6c28e1390afd288db52526f31b7f9a1253878: Status 404 returned error can't find the container with id ef84e1d63f578ee8bb0a56a4dde6c28e1390afd288db52526f31b7f9a1253878
Sep 06 05:54:09.967826 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:09.967775    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c6cf0e_08b2_4db0_9e50_6ebfec68eff3.slice/crio-19a4889beefa7973c5c29ee1b133d3547fce7e5d06b4e1eb995c4ca825f51348 WatchSource:0}: Error finding container 19a4889beefa7973c5c29ee1b133d3547fce7e5d06b4e1eb995c4ca825f51348: Status 404 returned error can't find the container with id 19a4889beefa7973c5c29ee1b133d3547fce7e5d06b4e1eb995c4ca825f51348
Sep 06 05:54:10.314429 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.314142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:10.314569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.314275    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:10.314569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.314463    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:10.314569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.314475    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:10.314569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.314525    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:12.314509767 +0000 UTC m=+6.018062483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:10.415156 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.415111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:10.415156 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.415167    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:10.415402 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.415293    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:10.415402 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.415344    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:12.415327399 +0000 UTC m=+6.118880115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:10.415402 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.415291    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:10.415402 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.415387    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:12.415375764 +0000 UTC m=+6.118928479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:10.717049 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.716630    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 05:49:09 +0000 UTC" deadline="2028-02-14 11:43:39.936239655 +0000 UTC"
Sep 06 05:54:10.717049 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.716688    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12629h49m29.219556199s"
Sep 06 05:54:10.787165 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.785905    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:10.787165 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.786047    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:10.787165 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.786494    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:10.787165 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:10.786606    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:10.821309 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.821272    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwl4k" event={"ID":"e6aa072d-833e-4dbf-b55f-6e7b6951e690","Type":"ContainerStarted","Data":"64cd4ff6dbeee2ce912b3cb7b327c0fe3878819db2893296255f5f6b90bd32d4"}
Sep 06 05:54:10.829164 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.828590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal" event={"ID":"00d162c63e419062586c3987b0b951dc","Type":"ContainerStarted","Data":"52b23c5b9fe584269559f60b12499a624731d0a17c5b1ff2130cc9cd505a43e1"}
Sep 06 05:54:10.835154 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.835037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cmg6t" event={"ID":"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3","Type":"ContainerStarted","Data":"19a4889beefa7973c5c29ee1b133d3547fce7e5d06b4e1eb995c4ca825f51348"}
Sep 06 05:54:10.840477 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.840433    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfp5n" event={"ID":"07e0569e-4ba8-44c8-9dd3-10a7823e0848","Type":"ContainerStarted","Data":"ab370613b4448f55ab80f4e05b9ce93df35fc38f58fdd47d4f5c4d5e6fa77489"}
Sep 06 05:54:10.843549 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.843496    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-33.ec2.internal" podStartSLOduration=1.843476348 podStartE2EDuration="1.843476348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:10.842677217 +0000 UTC m=+4.546229957" watchObservedRunningTime="2026-09-06 05:54:10.843476348 +0000 UTC m=+4.547029089"
Sep 06 05:54:10.847010 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.846909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"41fb3fa0547dfa67d937cb9b3f96f3d600d366af8793c795234967027034c622"}
Sep 06 05:54:10.854809 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.854748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdf8l" event={"ID":"d8ee0da7-abb2-4bcd-b504-6487a14049d6","Type":"ContainerStarted","Data":"97aeb6a83ab2355fb950da7c2c3dcb6eb067fb63e55bc96f4084e3f4116fdbca"}
Sep 06 05:54:10.868263 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.867612    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ecdb99928c19c8ca2cd1bded8a5a374" containerID="de8ef0386cba78a080527ee29f21b307d5e5ba1c24b0d33db7534eb796fda445" exitCode=0
Sep 06 05:54:10.868263 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.868203    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal" event={"ID":"2ecdb99928c19c8ca2cd1bded8a5a374","Type":"ContainerDied","Data":"de8ef0386cba78a080527ee29f21b307d5e5ba1c24b0d33db7534eb796fda445"}
Sep 06 05:54:10.883594 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.882771    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerStarted","Data":"ef84e1d63f578ee8bb0a56a4dde6c28e1390afd288db52526f31b7f9a1253878"}
Sep 06 05:54:10.898376 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.898347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kdpv" event={"ID":"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25","Type":"ContainerStarted","Data":"60c2bea834e8d94bea422d4b2da288df4a2b9ae198750aac9e83498a90e75097"}
Sep 06 05:54:10.914429 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.914346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4" event={"ID":"98bcb82f-59d2-4f9a-8ac4-27201d22a233","Type":"ContainerStarted","Data":"6a4355a09eeee2e0bd054581b6c7306ccfb68a58371e7ba698f1032b020cf23f"}
Sep 06 05:54:10.925028 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:10.924716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sq2fh" event={"ID":"0c060475-db7c-41ec-ac4b-5840c313ef83","Type":"ContainerStarted","Data":"1c09f27ada2f8b7420bb137e1ae9459b7af6c4da52b9d662a8f9fbcb237c81d6"}
Sep 06 05:54:11.781431 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:11.781289    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:11.781431 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:11.781419    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:11.937378 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:11.937222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal" event={"ID":"2ecdb99928c19c8ca2cd1bded8a5a374","Type":"ContainerStarted","Data":"11be5fd597537a6e1a97316a2d128ba2af6ab0b09fc27108a840a29c8a0b29dd"}
Sep 06 05:54:12.330680 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:12.330604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:12.330838 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.330779    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:12.330838 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.330803    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:12.330838 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.330818    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:12.330998 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.330880    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:16.330860663 +0000 UTC m=+10.034413385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:12.431477 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:12.431432    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:12.431477 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:12.431491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:12.431732 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.431675    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:12.431803 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.431741    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:16.431719313 +0000 UTC m=+10.135272055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:12.432180 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.432161    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:12.432287 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.432213    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:16.432197913 +0000 UTC m=+10.135750634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:12.780713 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:12.780618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:12.780877 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.780776    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:12.780936 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:12.780873    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:12.780997 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:12.780970    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:13.780932 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:13.780893    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:13.781493 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:13.781041    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:14.781570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:14.780601    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:14.781570 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:14.780796    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:14.781570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:14.781257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:14.781570 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:14.781397    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:15.781380 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:15.781304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:15.781552 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:15.781440    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:16.364665 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:16.364618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:16.365317 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.364828    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:16.365317 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.364857    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:16.365317 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.364872    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:16.365317 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.364938    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:24.364916793 +0000 UTC m=+18.068469513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:16.465395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:16.465449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.465594    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.465664    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:24.465643723 +0000 UTC m=+18.169196454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.466082    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:16.466186 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.466133    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:24.46611754 +0000 UTC m=+18.169670264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:16.782569 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:16.781604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:16.782569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.781707    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:16.782569 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:16.782356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:16.782569 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:16.782466    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:17.780723 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:17.780685    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:17.781239 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:17.780831    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:18.785536 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:18.784768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:18.785536 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:18.784903    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:18.785536 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:18.785382    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:18.785536 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:18.785499    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:19.780921 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:19.780882    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:19.781179 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:19.781021    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:20.780706 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:20.780666    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:20.781264 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:20.780718    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:20.781264 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:20.780829    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:20.781264 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:20.780950    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:21.780946 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:21.780913    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:21.781410 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:21.781053    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:22.788167 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:22.788134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:22.788595 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:22.788247    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:22.788595 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:22.788302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:22.788595 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:22.788403    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:23.780483 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:23.780449    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:23.780709 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:23.780555    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:24.432517 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:24.432475    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:24.432989 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.432630    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:24.432989 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.432658    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:24.432989 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.432672    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:24.432989 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.432736    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:40.432719436 +0000 UTC m=+34.136272172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:24.532853 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:24.532814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:24.533012 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:24.532863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:24.533012 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.532981    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:24.533152 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.533041    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:40.53302676 +0000 UTC m=+34.236579475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:24.533152 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.532981    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:24.533152 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.533139    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:54:40.533125229 +0000 UTC m=+34.236677943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:24.780664 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:24.780582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:24.780824 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.780700    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:24.780824 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:24.780728    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:24.780824 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:24.780787    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:25.780868 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:25.780829    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:25.781263 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:25.780964    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:26.781756 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:26.781683    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:26.782252 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:26.781765    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:26.782252 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:26.781868    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:26.782252 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:26.782015    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:27.780633 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:27.780599    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:27.780790 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:27.780715    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:28.781006 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.780817    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:28.781481 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.780845    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:28.781481 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:28.781121    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:28.781481 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:28.781151    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:28.985721 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.985673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdf8l" event={"ID":"d8ee0da7-abb2-4bcd-b504-6487a14049d6","Type":"ContainerStarted","Data":"91200ca9c831133aae340f574ee4015d0c2823ae6a5a7cacaad0248160d12b70"}
Sep 06 05:54:28.987574 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.987544    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="e1feb4f3a84294ad8f6eff4a7ef75282cc32d77df8737939121c4f78ab356f01" exitCode=0
Sep 06 05:54:28.987726 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.987634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"e1feb4f3a84294ad8f6eff4a7ef75282cc32d77df8737939121c4f78ab356f01"}
Sep 06 05:54:28.989439 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.989413    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kdpv" event={"ID":"b23f16e8-d43a-4b61-bb80-b7e0c39d5c25","Type":"ContainerStarted","Data":"4e89a283c404412829ada0a30731f4ed4a23c2f303148a635d7b34c472f05575"}
Sep 06 05:54:28.991191 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.991159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4" event={"ID":"98bcb82f-59d2-4f9a-8ac4-27201d22a233","Type":"ContainerStarted","Data":"03e442fccae11a9a83af025635604e53705b61ffc0e2499bb48539dfc242afba"}
Sep 06 05:54:28.992911 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.992889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sq2fh" event={"ID":"0c060475-db7c-41ec-ac4b-5840c313ef83","Type":"ContainerStarted","Data":"26886734dd3b53700ae00471849f24803660652400f5397858b1074fa450e7cf"}
Sep 06 05:54:28.994780 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.994758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwl4k" event={"ID":"e6aa072d-833e-4dbf-b55f-6e7b6951e690","Type":"ContainerStarted","Data":"4ff6ce7be06c0f14a64a0e2bddaa4a878599f79c8e5ae864ff6910c92b68ea6e"}
Sep 06 05:54:28.996402 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.996378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cmg6t" event={"ID":"98c6cf0e-08b2-4db0-9e50-6ebfec68eff3","Type":"ContainerStarted","Data":"98b14119f2c711623419fc5d9391e3ee30d4beae5c6cd32c2635dcff42367675"}
Sep 06 05:54:28.999374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.999329    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fdf8l" podStartSLOduration=3.703120109 podStartE2EDuration="21.999313936s" totalImagesPullingTime="18.296193827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:28.999218817 +0000 UTC m=+22.702771552" watchObservedRunningTime="2026-09-06 05:54:28.999313936 +0000 UTC m=+22.702866675"
Sep 06 05:54:28.999861 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:28.999830    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-33.ec2.internal" podStartSLOduration=19.999822101 podStartE2EDuration="19.999822101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:11.954456893 +0000 UTC m=+5.658009633" watchObservedRunningTime="2026-09-06 05:54:28.999822101 +0000 UTC m=+22.703374838"
Sep 06 05:54:29.000028 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000010    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"808e9b9b258cec966b5496ecd7dceae9dc9fcb81be8569c332880ace563c03c7"}
Sep 06 05:54:29.000128 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"d27136d5d7fd1bef5fb2ed14095f2ce8790088a1f5ead9fae8c7c2a9d5a6b125"}
Sep 06 05:54:29.000194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"0cd9a7f9ede89437ec7894f7d5097dedda4a2ba4d9616337a779daddab40968b"}
Sep 06 05:54:29.000252 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000201    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"b804fa41af09e2a140ae27f3dcc9a7264d0f27fbc69aa65afb411e2382d74abb"}
Sep 06 05:54:29.000306 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000250    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"b5e5c475a67ff33afb8ae5471ad3dc8cd5d907490b483a66b15d1dcc7a1e691c"}
Sep 06 05:54:29.000306 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.000297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"6a141d15e87b025642decf84daffebb6b04ab520c2b00cac40768dd1f163d1fb"}
Sep 06 05:54:29.015559 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.015506    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8kdpv" podStartSLOduration=10.891628324 podStartE2EDuration="22.015485436s" totalImagesPullingTime="11.123857112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:29.014865544 +0000 UTC m=+22.718418318" watchObservedRunningTime="2026-09-06 05:54:29.015485436 +0000 UTC m=+22.719038176"
Sep 06 05:54:29.031438 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.031390    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zwl4k" podStartSLOduration=3.72466464 podStartE2EDuration="22.031373954s" totalImagesPullingTime="18.306709314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:29.03074966 +0000 UTC m=+22.734302395" watchObservedRunningTime="2026-09-06 05:54:29.031373954 +0000 UTC m=+22.734926690"
Sep 06 05:54:29.046181 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.046081    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cmg6t" podStartSLOduration=3.696395897 podStartE2EDuration="22.046047886s" totalImagesPullingTime="18.349651989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:29.045434188 +0000 UTC m=+22.748986925" watchObservedRunningTime="2026-09-06 05:54:29.046047886 +0000 UTC m=+22.749600624"
Sep 06 05:54:29.097587 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.097530    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sq2fh" podStartSLOduration=3.795462214 podStartE2EDuration="22.09750879s" totalImagesPullingTime="18.302046576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:29.097271431 +0000 UTC m=+22.800824169" watchObservedRunningTime="2026-09-06 05:54:29.09750879 +0000 UTC m=+22.801061527"
Sep 06 05:54:29.100257 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.100226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:29.162545 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.162510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:29.263208 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.263176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:29.447539 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.447506    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 05:54:29.556979 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.556941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:29.735327 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.735193    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T05:54:29.44752332Z","UUID":"fafc0c28-6476-4228-933b-8ba87e2d1dfa","Handler":null,"Name":"","Endpoint":""}
Sep 06 05:54:29.736716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.736692    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 05:54:29.736716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.736725    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 05:54:29.780225 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:29.780199    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:29.780370 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:29.780355    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:30.008445 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.006308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4" event={"ID":"98bcb82f-59d2-4f9a-8ac4-27201d22a233","Type":"ContainerStarted","Data":"7ad3c750c322b8e5ae9416871e67ae06a0c5e4064d9f4bcf7145911b70704287"}
Sep 06 05:54:30.008445 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.008365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfp5n" event={"ID":"07e0569e-4ba8-44c8-9dd3-10a7823e0848","Type":"ContainerStarted","Data":"a4efcad33536091a31e5582bdd47037c6ed0f4f8c518fbcec9ca5fd3afbf1618"}
Sep 06 05:54:30.024471 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.024416    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tfp5n" podStartSLOduration=4.734138632 podStartE2EDuration="23.024398315s" totalImagesPullingTime="18.290259683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:30.024240202 +0000 UTC m=+23.727792939" watchObservedRunningTime="2026-09-06 05:54:30.024398315 +0000 UTC m=+23.727951052"
Sep 06 05:54:30.142197 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.142146    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:30.142468 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.142448    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:30.142995 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.142972    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:30.242605 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.242551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:30.781313 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.781239    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:30.781477 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:30.781238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:30.781477 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:30.781357    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:30.781477 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:30.781428    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:31.013144 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.013105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4" event={"ID":"98bcb82f-59d2-4f9a-8ac4-27201d22a233","Type":"ContainerStarted","Data":"d6f0e3c7c7b76391f44fb9531164f87a537326fb69afc4bebd8a0f2b9fdf86f0"}
Sep 06 05:54:31.017596 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.017559    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"0d828f7bafed04f35cee8df0665ca421f5d1e0ab75ba0c61f6cad032bb4afa9c"}
Sep 06 05:54:31.017916 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.017894    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:31.018599 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.018568    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8kdpv"
Sep 06 05:54:31.030478 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.030426    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdsv4" podStartSLOduration=3.917943122 podStartE2EDuration="24.030408464s" totalImagesPullingTime="20.112465342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:31.028719763 +0000 UTC m=+24.732272499" watchObservedRunningTime="2026-09-06 05:54:31.030408464 +0000 UTC m=+24.733961199"
Sep 06 05:54:31.095755 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.095668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:31.179881 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.179847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:31.281213 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.281182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:31.781210 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:31.781137    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:31.781418 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:31.781276    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:32.781175 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:32.781137    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:32.781752 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:32.781137    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:32.781752 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:32.781282    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:32.781752 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:32.781385    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:33.612843 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:33.612653    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:33.714948 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:33.714914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:33.781256 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:33.781217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:33.781798 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:33.781336    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:34.027747 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.027662    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" event={"ID":"03024c88-a2e9-4f74-9e27-c48e8cb493f6","Type":"ContainerStarted","Data":"3379c9aeb024c730ef779199f4323b5f3ed384a2b211a4588af9c5cef0070ed6"}
Sep 06 05:54:34.028021 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.027998    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:34.029594 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.029564    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="d5204f7f5a295ceca24d3df3f0cfc0db0d84d94699e8ce1399b375c1461be1a9" exitCode=0
Sep 06 05:54:34.029705 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.029603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"d5204f7f5a295ceca24d3df3f0cfc0db0d84d94699e8ce1399b375c1461be1a9"}
Sep 06 05:54:34.043538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.043517    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:34.057610 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.057572    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4" podStartSLOduration=8.700744125 podStartE2EDuration="27.057559377s" totalImagesPullingTime="18.356815252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:34.057371984 +0000 UTC m=+27.760924719" watchObservedRunningTime="2026-09-06 05:54:34.057559377 +0000 UTC m=+27.761112200"
Sep 06 05:54:34.784533 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.784378    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:34.784907 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.784433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:34.784907 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:34.784603    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:34.784907 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:34.784720    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:34.982616 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.982537    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wsczd"]
Sep 06 05:54:34.985408 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.985384    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nk5nq"]
Sep 06 05:54:34.985507 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.985490    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:34.985646 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:34.985597    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:34.996379 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:34.996316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pvwwb"]
Sep 06 05:54:35.034516 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.034483    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="bf0b0f6a85fbfaa0a6fc15d16ccf5745f865611971d5d22a486db27622153d56" exitCode=0
Sep 06 05:54:35.034668 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.034578    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"bf0b0f6a85fbfaa0a6fc15d16ccf5745f865611971d5d22a486db27622153d56"}
Sep 06 05:54:35.034740 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.034710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:35.034843 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:35.034822    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:35.034904 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.034864    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:35.035345 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.035237    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:35.035345 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:35.035324    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:35.035437 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.035362    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:35.050263 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.050238    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:54:35.635300 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.635269    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:35.682708 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:35.682676    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:36.039534 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:36.039504    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="15590ac18a40f0a36febe670c81f955f6fa32631d878ccf2f4988e7daaf401fd" exitCode=0
Sep 06 05:54:36.040384 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:36.039592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"15590ac18a40f0a36febe670c81f955f6fa32631d878ccf2f4988e7daaf401fd"}
Sep 06 05:54:36.783600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:36.783568    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:36.783600 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:36.783590    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:36.783813 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:36.783590    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:36.783813 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:36.783671    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:36.783813 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:36.783764    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:36.783923 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:36.783833    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:38.784152 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:38.783964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:38.784585 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:38.784030    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:38.784585 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:38.784238    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:38.784585 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:38.784034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:38.784585 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:38.784309    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:38.784585 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:38.784407    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:40.451893 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.451866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:40.452305 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.452013    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 05:54:40.452305 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.452035    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 05:54:40.452305 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.452045    2567 projected.go:196] Error preparing data for projected volume kube-api-access-bkkpn for pod openshift-network-diagnostics/network-check-target-nk5nq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:40.452305 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.452114    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn podName:6ad16abd-5f82-4fec-af5a-3b731d4114c1 nodeName:}" failed. No retries permitted until 2026-09-06 05:55:12.452097095 +0000 UTC m=+66.155649809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkkpn" (UniqueName: "kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn") pod "network-check-target-nk5nq" (UID: "6ad16abd-5f82-4fec-af5a-3b731d4114c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 05:54:40.553102 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.553069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:40.553245 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.553108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:40.553245 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.553209    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:40.553324 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.553258    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:55:12.553244602 +0000 UTC m=+66.256797316 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 05:54:40.553324 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.553209    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:40.553395 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.553327    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret podName:8eeb7275-b0a1-4dca-bc64-979136db13eb nodeName:}" failed. No retries permitted until 2026-09-06 05:55:12.553315575 +0000 UTC m=+66.256868307 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret") pod "global-pull-secret-syncer-wsczd" (UID: "8eeb7275-b0a1-4dca-bc64-979136db13eb") : object "kube-system"/"original-pull-secret" not registered
Sep 06 05:54:40.786297 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.786269    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:40.786409 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.786303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:40.786409 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.786383    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsczd" podUID="8eeb7275-b0a1-4dca-bc64-979136db13eb"
Sep 06 05:54:40.786510 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:40.786441    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:40.786577 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.786558    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nk5nq" podUID="6ad16abd-5f82-4fec-af5a-3b731d4114c1"
Sep 06 05:54:40.786671 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:40.786655    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:54:41.054321 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.054273    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="e0976fb9474b86ca334a7fb463e97532470d5eef00ba4f5ed9fbcdecb16d52c9" exitCode=0
Sep 06 05:54:41.054466 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.054326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"e0976fb9474b86ca334a7fb463e97532470d5eef00ba4f5ed9fbcdecb16d52c9"}
Sep 06 05:54:41.135637 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.135616    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-33.ec2.internal" event="NodeReady"
Sep 06 05:54:41.135758 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.135747    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 05:54:41.176700 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.176680    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zlcmz"]
Sep 06 05:54:41.191848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.191825    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b96bw"]
Sep 06 05:54:41.192309 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.192287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.194715 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.194694    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 05:54:41.195040 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.195019    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 05:54:41.195161 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.195105    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ppscw\""
Sep 06 05:54:41.216117 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.216092    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlcmz"]
Sep 06 05:54:41.216201 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.216140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b96bw"]
Sep 06 05:54:41.216201 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.216154    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.218821 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.218797    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 05:54:41.218821 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.218814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 05:54:41.218821 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.218822    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 05:54:41.219010 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.218895    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hqtmv\""
Sep 06 05:54:41.287403 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.287373    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:41.336824 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.336727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:41.360688 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1df33f75-f9ac-4088-a708-5e570b02dea5-cert\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.360762 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.360762 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9cp5\" (UniqueName: \"kubernetes.io/projected/1df33f75-f9ac-4088-a708-5e570b02dea5-kube-api-access-w9cp5\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.360834 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99895669-efaf-48a4-a29d-6ab0667aa167-tmp-dir\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.360834 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntqbw\" (UniqueName: \"kubernetes.io/projected/99895669-efaf-48a4-a29d-6ab0667aa167-kube-api-access-ntqbw\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.360834 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.360815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99895669-efaf-48a4-a29d-6ab0667aa167-config-volume\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462010 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.461980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1df33f75-f9ac-4088-a708-5e570b02dea5-cert\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462053    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9cp5\" (UniqueName: \"kubernetes.io/projected/1df33f75-f9ac-4088-a708-5e570b02dea5-kube-api-access-w9cp5\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99895669-efaf-48a4-a29d-6ab0667aa167-tmp-dir\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntqbw\" (UniqueName: \"kubernetes.io/projected/99895669-efaf-48a4-a29d-6ab0667aa167-kube-api-access-ntqbw\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:41.462163    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99895669-efaf-48a4-a29d-6ab0667aa167-config-volume\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462425 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:41.462224    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:41.962206846 +0000 UTC m=+35.665759567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:54:41.462715 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462478    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99895669-efaf-48a4-a29d-6ab0667aa167-tmp-dir\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.462715 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.462668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99895669-efaf-48a4-a29d-6ab0667aa167-config-volume\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.466198 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.466178    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1df33f75-f9ac-4088-a708-5e570b02dea5-cert\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.470655 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.470629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntqbw\" (UniqueName: \"kubernetes.io/projected/99895669-efaf-48a4-a29d-6ab0667aa167-kube-api-access-ntqbw\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.470722 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.470674    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9cp5\" (UniqueName: \"kubernetes.io/projected/1df33f75-f9ac-4088-a708-5e570b02dea5-kube-api-access-w9cp5\") pod \"ingress-canary-b96bw\" (UID: \"1df33f75-f9ac-4088-a708-5e570b02dea5\") " pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.524760 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.524736    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b96bw"
Sep 06 05:54:41.825424 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.825395    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b96bw"]
Sep 06 05:54:41.829311 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:54:41.829284    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df33f75_f9ac_4088_a708_5e570b02dea5.slice/crio-044e4f4da85641b23684f8b7627beeea9cf43aef1f3ebfe774ff0843cad46561 WatchSource:0}: Error finding container 044e4f4da85641b23684f8b7627beeea9cf43aef1f3ebfe774ff0843cad46561: Status 404 returned error can't find the container with id 044e4f4da85641b23684f8b7627beeea9cf43aef1f3ebfe774ff0843cad46561
Sep 06 05:54:41.965712 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:41.965483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:41.965712 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:41.965643    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:41.965889 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:41.965725    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:42.965707536 +0000 UTC m=+36.669260250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:54:42.057325 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.057292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b96bw" event={"ID":"1df33f75-f9ac-4088-a708-5e570b02dea5","Type":"ContainerStarted","Data":"044e4f4da85641b23684f8b7627beeea9cf43aef1f3ebfe774ff0843cad46561"}
Sep 06 05:54:42.060602 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.060576    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa8e1de9-0276-49d2-ad3c-4c4e79826925" containerID="fb63c2f3d175f2a2468a27e1848089a5c86edb1c5189b21353ca62756284cf4c" exitCode=0
Sep 06 05:54:42.060699 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.060612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerDied","Data":"fb63c2f3d175f2a2468a27e1848089a5c86edb1c5189b21353ca62756284cf4c"}
Sep 06 05:54:42.780941 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.780908    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:54:42.780941 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.780927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:54:42.781448 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.781011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:54:42.784791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.784767    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9n5z\""
Sep 06 05:54:42.784924 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.784807    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 05:54:42.784924 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.784829    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rnw9\""
Sep 06 05:54:42.785025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.784969    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 05:54:42.785025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.784996    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 05:54:42.785148 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.785049    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 05:54:42.973181 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:42.973145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:42.973354 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:42.973265    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:42.973354 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:42.973338    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:44.973316741 +0000 UTC m=+38.676869469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:54:43.067582 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:43.067551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" event={"ID":"fa8e1de9-0276-49d2-ad3c-4c4e79826925","Type":"ContainerStarted","Data":"b908502e897032d92c49c44e49837f714fee14b948237a244a31b3cc968b6ddf"}
Sep 06 05:54:43.088777 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:43.088724    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j4ktq" podStartSLOduration=14.934697593 podStartE2EDuration="36.088704116s" totalImagesPullingTime="21.154006523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:43.086545463 +0000 UTC m=+36.790098198" watchObservedRunningTime="2026-09-06 05:54:43.088704116 +0000 UTC m=+36.792256854"
Sep 06 05:54:43.205423 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:43.205394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:43.306390 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:43.306357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:44.989879 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:44.989841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:44.990502 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:44.989959    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:44.990502 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:44.990010    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:48.989995421 +0000 UTC m=+42.693548135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:54:45.074042 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:45.074006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b96bw" event={"ID":"1df33f75-f9ac-4088-a708-5e570b02dea5","Type":"ContainerStarted","Data":"21a6f9f3c2f602b7e56301539736bc23da0d1ef4ac42a2ed2008a44481d0ae03"}
Sep 06 05:54:45.089746 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:45.089701    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b96bw" podStartSLOduration=1.632322837 podStartE2EDuration="4.089684676s" totalImagesPullingTime="2.457361839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:54:45.089261574 +0000 UTC m=+38.792814309" watchObservedRunningTime="2026-09-06 05:54:45.089684676 +0000 UTC m=+38.793237409"
Sep 06 05:54:46.000336 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:46.000302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:46.100973 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:46.100939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:49.018620 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:49.018579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:49.018992 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:49.018724    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:49.018992 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:49.018792    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:54:57.018774322 +0000 UTC m=+50.722327038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:54:55.167954 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:55.167918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:55.270184 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:55.270152    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:55.586795 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:55.586768    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:55.687634 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:55.687607    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:54:57.071259 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:54:57.071220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:54:57.071636 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:57.071344    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:54:57.071636 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:54:57.071399    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:55:13.071384916 +0000 UTC m=+66.774937630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:55:07.053388 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:07.053357    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx2k4"
Sep 06 05:55:07.160828 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:07.160796    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:55:07.214200 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:07.214171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:55:12.469485 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.469431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:55:12.472421 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.472400    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 05:55:12.482546 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.482524    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 05:55:12.493732 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.493704    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkkpn\" (UniqueName: \"kubernetes.io/projected/6ad16abd-5f82-4fec-af5a-3b731d4114c1-kube-api-access-bkkpn\") pod \"network-check-target-nk5nq\" (UID: \"6ad16abd-5f82-4fec-af5a-3b731d4114c1\") " pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:55:12.570013 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.569974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:55:12.570013 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.570019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:55:12.572846 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.572827    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 05:55:12.572905 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.572872    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 05:55:12.581034 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:12.581017    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 05:55:12.581128 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:12.581094    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:56:16.581079637 +0000 UTC m=+130.284632350 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : secret "metrics-daemon-secret" not found
Sep 06 05:55:12.583076 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.583041    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eeb7275-b0a1-4dca-bc64-979136db13eb-original-pull-secret\") pod \"global-pull-secret-syncer-wsczd\" (UID: \"8eeb7275-b0a1-4dca-bc64-979136db13eb\") " pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:55:12.796421 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.796343    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9n5z\""
Sep 06 05:55:12.799390 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.799377    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsczd"
Sep 06 05:55:12.803824 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:12.803809    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:55:13.068584 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:13.068555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wsczd"]
Sep 06 05:55:13.071535 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:55:13.071508    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eeb7275_b0a1_4dca_bc64_979136db13eb.slice/crio-c8632df39c3ee4d362d4cb4de96b831a38322e17bde67db4d35438e7472aafdf WatchSource:0}: Error finding container c8632df39c3ee4d362d4cb4de96b831a38322e17bde67db4d35438e7472aafdf: Status 404 returned error can't find the container with id c8632df39c3ee4d362d4cb4de96b831a38322e17bde67db4d35438e7472aafdf
Sep 06 05:55:13.073415 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:13.073386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:55:13.074449 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:13.074242    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:55:13.074449 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:13.074312    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:55:45.074290627 +0000 UTC m=+98.777843347 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:55:13.074946 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:13.074926    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nk5nq"]
Sep 06 05:55:13.077992 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:55:13.077969    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad16abd_5f82_4fec_af5a_3b731d4114c1.slice/crio-7a4bece47123622fdadcdce22b431db1263667a8270265573ec4da5547a08826 WatchSource:0}: Error finding container 7a4bece47123622fdadcdce22b431db1263667a8270265573ec4da5547a08826: Status 404 returned error can't find the container with id 7a4bece47123622fdadcdce22b431db1263667a8270265573ec4da5547a08826
Sep 06 05:55:13.139783 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:13.139753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wsczd" event={"ID":"8eeb7275-b0a1-4dca-bc64-979136db13eb","Type":"ContainerStarted","Data":"c8632df39c3ee4d362d4cb4de96b831a38322e17bde67db4d35438e7472aafdf"}
Sep 06 05:55:13.140938 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:13.140917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nk5nq" event={"ID":"6ad16abd-5f82-4fec-af5a-3b731d4114c1","Type":"ContainerStarted","Data":"7a4bece47123622fdadcdce22b431db1263667a8270265573ec4da5547a08826"}
Sep 06 05:55:18.155853 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:18.155800    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nk5nq" event={"ID":"6ad16abd-5f82-4fec-af5a-3b731d4114c1","Type":"ContainerStarted","Data":"0b2406ab90ce44a87ce4c07ca9e7256a5ed7028d5ffb5540848341bbb976795e"}
Sep 06 05:55:18.156310 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:18.155969    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:55:18.157262 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:18.157241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wsczd" event={"ID":"8eeb7275-b0a1-4dca-bc64-979136db13eb","Type":"ContainerStarted","Data":"9c9679ec8c86a12fe79909f65d2f2bec8336db74f7a7fb3e7865fe4f32185f5c"}
Sep 06 05:55:18.171820 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:18.171779    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nk5nq" podStartSLOduration=67.031631541 podStartE2EDuration="1m11.171767102s" totalImagesPullingTime="4.140135561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:55:18.17053988 +0000 UTC m=+71.874092627" watchObservedRunningTime="2026-09-06 05:55:18.171767102 +0000 UTC m=+71.875319837"
Sep 06 05:55:18.184864 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:18.184824    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wsczd" podStartSLOduration=67.035563506 podStartE2EDuration="1m11.18481492s" totalImagesPullingTime="4.149251414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:55:18.183797984 +0000 UTC m=+71.887350719" watchObservedRunningTime="2026-09-06 05:55:18.18481492 +0000 UTC m=+71.888367656"
Sep 06 05:55:34.887524 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.887414    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"]
Sep 06 05:55:34.892377 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.892356    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"]
Sep 06 05:55:34.892513 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.892495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:34.895044 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.895019    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 06 05:55:34.895175 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.895031    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.895481 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.895465    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 06 05:55:34.895529 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.895469    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 06 05:55:34.896245 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.896229    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 06 05:55:34.897583 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.897566    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 06 05:55:34.897695 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.897678    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 06 05:55:34.897745 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.897711    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 06 05:55:34.897822 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.897806    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 06 05:55:34.900126 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.900103    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"]
Sep 06 05:55:34.903203 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.903177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"]
Sep 06 05:55:34.919806 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.919894 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5eab9971-ac63-487d-b7af-73fe0b73b732-klusterlet-config\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:34.919894 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919830    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.919894 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.919894 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzt8n\" (UniqueName: \"kubernetes.io/projected/5eab9971-ac63-487d-b7af-73fe0b73b732-kube-api-access-zzt8n\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:34.920130 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.920130 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa089ae-e429-42f8-baad-8abf3dc169c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.920130 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.919995    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m866\" (UniqueName: \"kubernetes.io/projected/afa089ae-e429-42f8-baad-8abf3dc169c0-kube-api-access-7m866\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:34.920130 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:34.920035    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eab9971-ac63-487d-b7af-73fe0b73b732-tmp\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.021170 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021140    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021170 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzt8n\" (UniqueName: \"kubernetes.io/projected/5eab9971-ac63-487d-b7af-73fe0b73b732-kube-api-access-zzt8n\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.021374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa089ae-e429-42f8-baad-8abf3dc169c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m866\" (UniqueName: \"kubernetes.io/projected/afa089ae-e429-42f8-baad-8abf3dc169c0-kube-api-access-7m866\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021374 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eab9971-ac63-487d-b7af-73fe0b73b732-tmp\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.021686 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.021686 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5eab9971-ac63-487d-b7af-73fe0b73b732-klusterlet-config\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.021813 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.021694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eab9971-ac63-487d-b7af-73fe0b73b732-tmp\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.022473 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.022442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa089ae-e429-42f8-baad-8abf3dc169c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.023794 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.023772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-ca\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.024079 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.024043    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.024160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.024144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.024160 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.024153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5eab9971-ac63-487d-b7af-73fe0b73b732-klusterlet-config\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.024291 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.024271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/afa089ae-e429-42f8-baad-8abf3dc169c0-hub\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.028855 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.028831    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzt8n\" (UniqueName: \"kubernetes.io/projected/5eab9971-ac63-487d-b7af-73fe0b73b732-kube-api-access-zzt8n\") pod \"klusterlet-addon-workmgr-846b7c8475-hqll5\" (UID: \"5eab9971-ac63-487d-b7af-73fe0b73b732\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.029193 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.029172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m866\" (UniqueName: \"kubernetes.io/projected/afa089ae-e429-42f8-baad-8abf3dc169c0-kube-api-access-7m866\") pod \"cluster-proxy-proxy-agent-d7cfbf956-4ql9w\" (UID: \"afa089ae-e429-42f8-baad-8abf3dc169c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.202931 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.202853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:35.217820 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.217797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 05:55:35.471973 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.471888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"]
Sep 06 05:55:35.475850 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:55:35.475813    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eab9971_ac63_487d_b7af_73fe0b73b732.slice/crio-c4ec77a46ef7855a003360f7e602029dd2f61e54d8d334da9668ad766730c938 WatchSource:0}: Error finding container c4ec77a46ef7855a003360f7e602029dd2f61e54d8d334da9668ad766730c938: Status 404 returned error can't find the container with id c4ec77a46ef7855a003360f7e602029dd2f61e54d8d334da9668ad766730c938
Sep 06 05:55:35.493044 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:35.492968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"]
Sep 06 05:55:35.495304 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:55:35.495278    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafa089ae_e429_42f8_baad_8abf3dc169c0.slice/crio-a822e376c7fe5fce137ba44dd19fc8d3b2d10a0be8a0572b4bfaf9c31810fa43 WatchSource:0}: Error finding container a822e376c7fe5fce137ba44dd19fc8d3b2d10a0be8a0572b4bfaf9c31810fa43: Status 404 returned error can't find the container with id a822e376c7fe5fce137ba44dd19fc8d3b2d10a0be8a0572b4bfaf9c31810fa43
Sep 06 05:55:36.197685 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:36.197644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerStarted","Data":"a822e376c7fe5fce137ba44dd19fc8d3b2d10a0be8a0572b4bfaf9c31810fa43"}
Sep 06 05:55:36.199612 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:36.199576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" event={"ID":"5eab9971-ac63-487d-b7af-73fe0b73b732","Type":"ContainerStarted","Data":"c4ec77a46ef7855a003360f7e602029dd2f61e54d8d334da9668ad766730c938"}
Sep 06 05:55:41.219345 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.219314    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerStarted","Data":"e9cd0c451ea39b8da99ed83cc75797e56fe20abd65abd9f4f1d431b326ebd880"}
Sep 06 05:55:41.219722 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.219350    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerStarted","Data":"49a4dead7e10362df3049317abd858cb765acca430c1aa767d48183f3d43ad06"}
Sep 06 05:55:41.220712 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.220677    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" event={"ID":"5eab9971-ac63-487d-b7af-73fe0b73b732","Type":"ContainerStarted","Data":"d42d62ca967cf19d68ecbe127ccd16372537709b335a7f3a7e223db05e9e78db"}
Sep 06 05:55:41.220920 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.220880    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:41.222460 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.222449    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:55:41.236436 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:41.236375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" podStartSLOduration=1.623099839 podStartE2EDuration="7.236196098s" totalImagesPullingTime="5.613096259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:55:41.236019157 +0000 UTC m=+94.939571893" watchObservedRunningTime="2026-09-06 05:55:41.236196098 +0000 UTC m=+94.939748848"
Sep 06 05:55:42.226519 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:42.226470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerStarted","Data":"5bd54f7c0c18325c576a720caf4d1b9b61b75a13d8c830b0679f34c5cc166b8e"}
Sep 06 05:55:42.244686 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:42.244635    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" podStartSLOduration=2.653410454 podStartE2EDuration="8.244618734s" totalImagesPullingTime="5.59120828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:55:42.243922726 +0000 UTC m=+95.947475469" watchObservedRunningTime="2026-09-06 05:55:42.244618734 +0000 UTC m=+95.948171507"
Sep 06 05:55:42.797619 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:42.797581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:55:42.899230 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:42.899193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:55:45.096910 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:45.096876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:55:45.097285 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:45.096985    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:55:45.097285 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:55:45.097036    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:56:49.097022583 +0000 UTC m=+162.800575297 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:55:49.162274 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:49.162239    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nk5nq"
Sep 06 05:55:57.297503 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:55:57.297466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:56:16.613810 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:56:16.613771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:56:16.614297 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:16.613917    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 05:56:16.614297 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:16.613984    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:18.613968088 +0000 UTC m=+252.317520801 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : secret "metrics-daemon-secret" not found
Sep 06 05:56:44.202245 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:44.202203    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zlcmz" podUID="99895669-efaf-48a4-a29d-6ab0667aa167"
Sep 06 05:56:44.398636 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:56:44.398606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:56:45.805693 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:45.805649    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:56:49.132012 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:56:49.131970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:56:49.132448 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:49.132109    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 05:56:49.132448 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:56:49.132159    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls podName:99895669-efaf-48a4-a29d-6ab0667aa167 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:51.132145603 +0000 UTC m=+284.835698316 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls") pod "dns-default-zlcmz" (UID: "99895669-efaf-48a4-a29d-6ab0667aa167") : secret "dns-default-metrics-tls" not found
Sep 06 05:56:57.781247 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:56:57.781211    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:57:41.221908 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:41.221865    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" podUID="5eab9971-ac63-487d-b7af-73fe0b73b732" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Sep 06 05:57:41.556603 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:41.556575    2567 generic.go:356] "Generic (PLEG): container finished" podID="5eab9971-ac63-487d-b7af-73fe0b73b732" containerID="d42d62ca967cf19d68ecbe127ccd16372537709b335a7f3a7e223db05e9e78db" exitCode=1
Sep 06 05:57:41.556760 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:41.556628    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" event={"ID":"5eab9971-ac63-487d-b7af-73fe0b73b732","Type":"ContainerDied","Data":"d42d62ca967cf19d68ecbe127ccd16372537709b335a7f3a7e223db05e9e78db"}
Sep 06 05:57:41.556963 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:41.556947    2567 scope.go:117] "RemoveContainer" containerID="d42d62ca967cf19d68ecbe127ccd16372537709b335a7f3a7e223db05e9e78db"
Sep 06 05:57:42.560936 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:42.560904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5" event={"ID":"5eab9971-ac63-487d-b7af-73fe0b73b732","Type":"ContainerStarted","Data":"b6ae691e1895eb6fcee72b4e5ac290f5035fb35bde39729a71d93066f32377a6"}
Sep 06 05:57:42.561340 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:42.561172    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:57:42.561803 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:57:42.561785    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-846b7c8475-hqll5"
Sep 06 05:58:18.644700 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:18.644659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:58:18.645249 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:18.644790    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 05:58:18.645249 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:18.644853    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs podName:4f1efb45-a0eb-4de5-a2d1-1557a281bab6 nodeName:}" failed. No retries permitted until 2026-09-06 06:00:20.644838671 +0000 UTC m=+374.348391385 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs") pod "network-metrics-daemon-pvwwb" (UID: "4f1efb45-a0eb-4de5-a2d1-1557a281bab6") : secret "metrics-daemon-secret" not found
Sep 06 05:58:22.839194 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:22.839141    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:22.926169 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:22.926135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:24.692210 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:24.692174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:24.740778 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:24.740740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:26.462283 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:26.462243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:26.564429 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:26.564399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:26.700907 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:26.700875    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:26.755968 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:26.755893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.257879 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.257846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.358263 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.358227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.530923 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.530847    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-26knw"]
Sep 06 05:58:27.533813 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.533793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.536512 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.536490    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5nmlz\""
Sep 06 05:58:27.536661 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.536639    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 05:58:27.536780 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.536681    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 05:58:27.536780 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.536700    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 05:58:27.537499 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.537482    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 05:58:27.546285 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.546260    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-26knw"]
Sep 06 05:58:27.555664 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.555641    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.602847 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.602825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjcd4\" (UniqueName: \"kubernetes.io/projected/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-api-access-hjcd4\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.602938 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.602857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.602938 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.602874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.603007 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.602956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7de92d74-d8f0-4ff5-881c-38fe64d20071-crio-socket\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.603007 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.602983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de92d74-d8f0-4ff5-881c-38fe64d20071-data-volume\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.658322 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.658291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.703647 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjcd4\" (UniqueName: \"kubernetes.io/projected/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-api-access-hjcd4\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703656    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7de92d74-d8f0-4ff5-881c-38fe64d20071-crio-socket\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de92d74-d8f0-4ff5-881c-38fe64d20071-data-volume\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:27.703754    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 05:58:27.703791 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:27.703774    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:27.704021 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:27.703850    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:28.203827314 +0000 UTC m=+261.907380028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:27.704021 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.703850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7de92d74-d8f0-4ff5-881c-38fe64d20071-crio-socket\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.704021 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:27.703877    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:28.203858809 +0000 UTC m=+261.907411546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : configmap "kube-rbac-proxy" not found
Sep 06 05:58:27.704021 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.704003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de92d74-d8f0-4ff5-881c-38fe64d20071-data-volume\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.712545 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.712528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjcd4\" (UniqueName: \"kubernetes.io/projected/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-api-access-hjcd4\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:27.747508 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.747485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.804633 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.804602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:27.905893 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:27.905871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:28.207221 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:28.207113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:28.207221 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:28.207167    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:28.207454 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:28.207252    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 05:58:28.207454 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:28.207289    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:28.207454 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:28.207329    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:29.207316796 +0000 UTC m=+262.910869510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:28.207454 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:28.207342    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:29.207334953 +0000 UTC m=+262.910887667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : configmap "kube-rbac-proxy" not found
Sep 06 05:58:28.798869 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:28.798833    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:29.214353 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:29.214315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:29.214353 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:29.214355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:29.214563 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:29.214422    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 05:58:29.214563 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:29.214472    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:31.21445705 +0000 UTC m=+264.918009769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : configmap "kube-rbac-proxy" not found
Sep 06 05:58:29.214563 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:29.214529    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:29.214563 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:29.214556    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:31.214548869 +0000 UTC m=+264.918101582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:29.465746 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:29.465671    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sq2fh_0c060475-db7c-41ec-ac4b-5840c313ef83/dns-node-resolver/0.log"
Sep 06 05:58:29.646387 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:29.646354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:30.665828 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:30.665795    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdf8l_d8ee0da7-abb2-4bcd-b504-6487a14049d6/node-ca/0.log"
Sep 06 05:58:30.796028 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:30.795992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:30.901244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:30.901214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:31.004340 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.004267    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:31.074442 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.074411    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b96bw_1df33f75-f9ac-4088-a708-5e570b02dea5/serve-healthcheck-canary/0.log"
Sep 06 05:58:31.102762 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.102736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:31.230622 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.230584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:31.230622 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.230618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:31.230831 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:31.230709    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 05:58:31.230831 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:31.230796    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:35.230765178 +0000 UTC m=+268.934317905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : configmap "kube-rbac-proxy" not found
Sep 06 05:58:31.230831 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:31.230801    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:31.230931 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:31.230844    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:35.230833832 +0000 UTC m=+268.934386546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:31.298265 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.298179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:31.398909 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.398876    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:31.498614 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:31.498584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:32.088153 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:32.088112    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:32.304865 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:32.304831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:32.700119 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:32.700081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:33.095025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:33.094997    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:33.492079 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:33.491972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:33.696943 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:33.696915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:33.749813 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:33.749725    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:34.095622 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:34.095585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:34.677622 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:34.677550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:34.726914 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:34.726883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:35.264284 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.264254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:35.264284 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.264290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:35.264697 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:35.264382    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 05:58:35.264697 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:35.264400    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:35.264697 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:35.264441    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:43.264422941 +0000 UTC m=+276.967975655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : configmap "kube-rbac-proxy" not found
Sep 06 05:58:35.264697 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:35.264456    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls podName:7de92d74-d8f0-4ff5-881c-38fe64d20071 nodeName:}" failed. No retries permitted until 2026-09-06 05:58:43.264449471 +0000 UTC m=+276.968002184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26knw" (UID: "7de92d74-d8f0-4ff5-881c-38fe64d20071") : secret "insights-runtime-extractor-tls" not found
Sep 06 05:58:35.301167 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.301144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:35.492028 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.491992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:35.543469 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.543393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:35.892353 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:35.892319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:36.098555 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:36.098518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:36.297869 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:36.297787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:36.406418 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:36.406366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:37.091544 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:37.091515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:37.285353 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:37.285321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:37.697309 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:37.697273    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:37.894355 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:37.894320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:38.098720 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:38.098691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:38.497302 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:38.497213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:39.295322 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:39.295284    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:39.695445 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:39.695404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:39.893692 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:39.893656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:39.941461 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:39.941428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:40.092682 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:40.092648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:40.498950 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:40.498864    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:40.892198 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:40.892168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:41.297510 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:41.297423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:41.495777 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:41.495742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:41.972862 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:41.972825    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:42.073520 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:42.073489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:42.158365 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:42.158333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:42.456129 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:42.456092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:43.320716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.320677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:43.320716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.320713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:43.321230 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.321211    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7de92d74-d8f0-4ff5-881c-38fe64d20071-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:43.323056 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.323022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7de92d74-d8f0-4ff5-881c-38fe64d20071-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26knw\" (UID: \"7de92d74-d8f0-4ff5-881c-38fe64d20071\") " pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:43.442462 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.442427    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-26knw"
Sep 06 05:58:43.708806 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.708733    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-26knw"]
Sep 06 05:58:43.711979 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:58:43.711945    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de92d74_d8f0_4ff5_881c_38fe64d20071.slice/crio-81d502eaf46d452e2300369611e8e411ce93d876fdc6d0d7d1c7512bdf4afad7 WatchSource:0}: Error finding container 81d502eaf46d452e2300369611e8e411ce93d876fdc6d0d7d1c7512bdf4afad7: Status 404 returned error can't find the container with id 81d502eaf46d452e2300369611e8e411ce93d876fdc6d0d7d1c7512bdf4afad7
Sep 06 05:58:43.731519 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:43.731490    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26knw" event={"ID":"7de92d74-d8f0-4ff5-881c-38fe64d20071","Type":"ContainerStarted","Data":"81d502eaf46d452e2300369611e8e411ce93d876fdc6d0d7d1c7512bdf4afad7"}
Sep 06 05:58:44.736538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:44.736449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26knw" event={"ID":"7de92d74-d8f0-4ff5-881c-38fe64d20071","Type":"ContainerStarted","Data":"16e3388a10f2b70b666fafc6b5cd56aeb8a581ef9536d1acb694bb9382f663ce"}
Sep 06 05:58:44.736538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:44.736511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26knw" event={"ID":"7de92d74-d8f0-4ff5-881c-38fe64d20071","Type":"ContainerStarted","Data":"9ec0a78a98353130bb6c9226e65615c134f608d26f8029282b14e810bab37d9b"}
Sep 06 05:58:46.744857 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:46.744813    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26knw" event={"ID":"7de92d74-d8f0-4ff5-881c-38fe64d20071","Type":"ContainerStarted","Data":"261de40a03eb78bd971f4899599c33ddac373a4db6f90b8ca0e99c6b8a9f1265"}
Sep 06 05:58:46.762748 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:46.762700    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-26knw" podStartSLOduration=17.65718241 podStartE2EDuration="19.762686131s" totalImagesPullingTime="2.105503721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:58:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:58:46.76167053 +0000 UTC m=+280.465223266" watchObservedRunningTime="2026-09-06 05:58:46.762686131 +0000 UTC m=+280.466238905"
Sep 06 05:58:47.398538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.398505    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c9d4bb797-nqfwm"]
Sep 06 05:58:47.399492 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:47.399469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zlcmz" podUID="99895669-efaf-48a4-a29d-6ab0667aa167"
Sep 06 05:58:47.401364 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.401349    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.404386 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.404367    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qcp4t\""
Sep 06 05:58:47.404685 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.404667    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 06 05:58:47.404889 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.404876    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 06 05:58:47.405650 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.405629    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 06 05:58:47.410245 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.410222    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 06 05:58:47.417320 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.417274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:47.417588 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.417568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c9d4bb797-nqfwm"]
Sep 06 05:58:47.510391 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.510355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:47.552747 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a08c080b-5ffe-4998-873e-ddef2b9e4607-ca-trust-extracted\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552791    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4k9w\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-kube-api-access-l4k9w\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-image-registry-private-configuration\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-installation-pull-secrets\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552948 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-bound-sa-token\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552948 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552876    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-tls\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552948 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552898    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-certificates\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.552948 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.552916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-trusted-ca\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653284 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-image-registry-private-configuration\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653284 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-installation-pull-secrets\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653284 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-bound-sa-token\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653465 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-tls\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653465 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-certificates\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653465 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-trusted-ca\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653465 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a08c080b-5ffe-4998-873e-ddef2b9e4607-ca-trust-extracted\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653465 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4k9w\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-kube-api-access-l4k9w\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.653881 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.653856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a08c080b-5ffe-4998-873e-ddef2b9e4607-ca-trust-extracted\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.654397 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.654372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-certificates\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.654496 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.654450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a08c080b-5ffe-4998-873e-ddef2b9e4607-trusted-ca\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.655856 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.655836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-image-registry-private-configuration\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.655856 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.655850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-registry-tls\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.655967 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.655876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a08c080b-5ffe-4998-873e-ddef2b9e4607-installation-pull-secrets\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.660994 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.660968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4k9w\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-kube-api-access-l4k9w\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.661186 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.661146    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a08c080b-5ffe-4998-873e-ddef2b9e4607-bound-sa-token\") pod \"image-registry-5c9d4bb797-nqfwm\" (UID: \"a08c080b-5ffe-4998-873e-ddef2b9e4607\") " pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.710744 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.710720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:47.748716 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.748664    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:58:47.982410 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:47.982325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c9d4bb797-nqfwm"]
Sep 06 05:58:47.985309 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:58:47.985287    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08c080b_5ffe_4998_873e_ddef2b9e4607.slice/crio-5791d023ac925763b524e0053989c72c226e87053efaf6352c21ecedf80614a9 WatchSource:0}: Error finding container 5791d023ac925763b524e0053989c72c226e87053efaf6352c21ecedf80614a9: Status 404 returned error can't find the container with id 5791d023ac925763b524e0053989c72c226e87053efaf6352c21ecedf80614a9
Sep 06 05:58:48.753421 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:48.753391    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm" event={"ID":"a08c080b-5ffe-4998-873e-ddef2b9e4607","Type":"ContainerStarted","Data":"dab3ffceedba885841a8f0bbe9dcf9473458407a226c2b2b778433fbb2cb0c03"}
Sep 06 05:58:48.753808 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:48.753437    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm" event={"ID":"a08c080b-5ffe-4998-873e-ddef2b9e4607","Type":"ContainerStarted","Data":"5791d023ac925763b524e0053989c72c226e87053efaf6352c21ecedf80614a9"}
Sep 06 05:58:48.753808 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:48.753468    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:58:48.774540 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:48.774489    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm" podStartSLOduration=1.774472957 podStartE2EDuration="1.774472957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:58:48.773646338 +0000 UTC m=+282.477199074" watchObservedRunningTime="2026-09-06 05:58:48.774472957 +0000 UTC m=+282.478025694"
Sep 06 05:58:50.159452 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:50.159411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:50.259898 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:50.259862    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:50.312234 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:50.312200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:50.658452 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:50.658418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:50.711482 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:50.711454    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:51.180405 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.180370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:58:51.182730 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.182705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99895669-efaf-48a4-a29d-6ab0667aa167-metrics-tls\") pod \"dns-default-zlcmz\" (UID: \"99895669-efaf-48a4-a29d-6ab0667aa167\") " pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:58:51.352205 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.352174    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ppscw\""
Sep 06 05:58:51.360133 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.360113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:58:51.631840 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.631766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlcmz"]
Sep 06 05:58:51.634319 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:58:51.634289    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99895669_efaf_48a4_a29d_6ab0667aa167.slice/crio-2c31b71183ecaf337be7c12e675d02f88b6465e59c046c546020ddcc98a88087 WatchSource:0}: Error finding container 2c31b71183ecaf337be7c12e675d02f88b6465e59c046c546020ddcc98a88087: Status 404 returned error can't find the container with id 2c31b71183ecaf337be7c12e675d02f88b6465e59c046c546020ddcc98a88087
Sep 06 05:58:51.763944 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:51.763910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlcmz" event={"ID":"99895669-efaf-48a4-a29d-6ab0667aa167","Type":"ContainerStarted","Data":"2c31b71183ecaf337be7c12e675d02f88b6465e59c046c546020ddcc98a88087"}
Sep 06 05:58:52.232479 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:52.232440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:53.771589 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:53.771555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlcmz" event={"ID":"99895669-efaf-48a4-a29d-6ab0667aa167","Type":"ContainerStarted","Data":"295bb4696ce35a1104a8a83ade995ba44bbbef9e4f05758640baa01ed83067b3"}
Sep 06 05:58:53.771979 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:53.771609    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlcmz" event={"ID":"99895669-efaf-48a4-a29d-6ab0667aa167","Type":"ContainerStarted","Data":"53f6f6d0792ed588e7866eb840b897a21d465796126775112b28e19b16c34074"}
Sep 06 05:58:53.771979 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:53.771677    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:58:53.787484 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:53.787437    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zlcmz" podStartSLOduration=251.380368832 podStartE2EDuration="4m12.78742313s" totalImagesPullingTime="1.407054298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:58:53.786562103 +0000 UTC m=+287.490114838" watchObservedRunningTime="2026-09-06 05:58:53.78742313 +0000 UTC m=+287.490975865"
Sep 06 05:58:54.307972 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.307937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.450479 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.450439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.541895 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.541858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.576285 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.576214    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2vpss"]
Sep 06 05:58:54.580722 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.580706    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.583235 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.583209    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 05:58:54.583386 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.583209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pdq8f\""
Sep 06 05:58:54.583386 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.583244    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 05:58:54.583386 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.583209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 05:58:54.583386 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.583305    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 05:58:54.584271 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.584250    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 05:58:54.584375 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.584272    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 05:58:54.615180 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.615154    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.710875 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.710847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711003 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.710891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711003 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.710909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-root\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711098 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.710997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-textfile\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711098 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.711032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmrw7\" (UniqueName: \"kubernetes.io/projected/f008550a-4455-48c0-b995-228efae0b97a-kube-api-access-gmrw7\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711098 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.711090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711195 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.711117    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-sys\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711195 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.711141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-wtmp\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.711195 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.711184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-metrics-client-ca\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.715122 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.715098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.811538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811507    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811538 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-root\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-textfile\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmrw7\" (UniqueName: \"kubernetes.io/projected/f008550a-4455-48c0-b995-228efae0b97a-kube-api-access-gmrw7\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:54.811663    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-sys\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:58:54.811718    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls podName:f008550a-4455-48c0-b995-228efae0b97a nodeName:}" failed. No retries permitted until 2026-09-06 05:58:55.311702414 +0000 UTC m=+289.015255127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls") pod "node-exporter-2vpss" (UID: "f008550a-4455-48c0-b995-228efae0b97a") : secret "node-exporter-tls" not found
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-sys\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-wtmp\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-metrics-client-ca\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-root\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.811933 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-wtmp\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.812471 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.811998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-textfile\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.812471 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.812386    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.812572 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.812537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f008550a-4455-48c0-b995-228efae0b97a-metrics-client-ca\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.813973 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.813957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.820688 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.820669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmrw7\" (UniqueName: \"kubernetes.io/projected/f008550a-4455-48c0-b995-228efae0b97a-kube-api-access-gmrw7\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:54.824205 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.824181    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:54.924321 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:54.924254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:55.024789 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.024755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:55.313675 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.313638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:55.315653 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.315634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:55.317916 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.317897    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f008550a-4455-48c0-b995-228efae0b97a-node-exporter-tls\") pod \"node-exporter-2vpss\" (UID: \"f008550a-4455-48c0-b995-228efae0b97a\") " pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:55.488854 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.488823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2vpss"
Sep 06 05:58:55.500528 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:58:55.500503    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf008550a_4455_48c0_b995_228efae0b97a.slice/crio-cd2d7386a1020b56928e1844f6b0ee094c723cd56631220f22d69664e69da24b WatchSource:0}: Error finding container cd2d7386a1020b56928e1844f6b0ee094c723cd56631220f22d69664e69da24b: Status 404 returned error can't find the container with id cd2d7386a1020b56928e1844f6b0ee094c723cd56631220f22d69664e69da24b
Sep 06 05:58:55.526349 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.526309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:55.606615 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.606553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:55.779056 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.779022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vpss" event={"ID":"f008550a-4455-48c0-b995-228efae0b97a","Type":"ContainerStarted","Data":"cd2d7386a1020b56928e1844f6b0ee094c723cd56631220f22d69664e69da24b"}
Sep 06 05:58:55.958838 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:55.958721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:56.783609 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:56.783525    2567 generic.go:356] "Generic (PLEG): container finished" podID="f008550a-4455-48c0-b995-228efae0b97a" containerID="d1c06f3c9cf23cb3e07b3b6774a4cac1b082d3e741e7b3cf1880b0c1236679b6" exitCode=0
Sep 06 05:58:56.784743 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:56.784717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vpss" event={"ID":"f008550a-4455-48c0-b995-228efae0b97a","Type":"ContainerDied","Data":"d1c06f3c9cf23cb3e07b3b6774a4cac1b082d3e741e7b3cf1880b0c1236679b6"}
Sep 06 05:58:57.469748 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:57.469707    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:57.788531 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:57.788455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vpss" event={"ID":"f008550a-4455-48c0-b995-228efae0b97a","Type":"ContainerStarted","Data":"bf45fc4fe1deb9478cadce72b9129401f46c76d91d92d56c73e8f980b6c9758a"}
Sep 06 05:58:57.788531 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:57.788500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vpss" event={"ID":"f008550a-4455-48c0-b995-228efae0b97a","Type":"ContainerStarted","Data":"4afaf0f92d9df0f9d71c16a3c306eaefe8ad18c6d6738d26d49cc2f9c5fca063"}
Sep 06 05:58:57.806778 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:57.806736    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2vpss" podStartSLOduration=2.913325767 podStartE2EDuration="3.806721503s" totalImagesPullingTime="834.377277ms" totalInitContainerRuntime="59.018459ms" isStatefulPod=true podCreationTimestamp="2026-09-06 05:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:58:57.805192911 +0000 UTC m=+291.508745647" watchObservedRunningTime="2026-09-06 05:58:57.806721503 +0000 UTC m=+291.510274239"
Sep 06 05:58:58.594845 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:58.594816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:59.331038 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:59.331001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:58:59.492932 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:58:59.492899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:00.755244 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:00.755199    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:00.782298 ip-10-0-129-33 kubenswrapper[2567]: E0906 05:59:00.782263    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pvwwb" podUID="4f1efb45-a0eb-4de5-a2d1-1557a281bab6"
Sep 06 05:59:01.250985 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:01.250954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:01.913628 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:01.913591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:02.014436 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:02.014404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:03.777936 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:03.777908    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zlcmz"
Sep 06 05:59:05.055101 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:05.055051    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:05.107915 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:05.107879    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:05.640008 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:05.639971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:05.738261 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:05.738232    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:06.699674 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:06.699650    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 05:59:09.760791 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:09.760764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c9d4bb797-nqfwm"
Sep 06 05:59:09.893402 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:09.893367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:09.995222 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:09.995189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:10.096671 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:10.096642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:10.194547 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:10.194518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:12.813688 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:12.813656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:12.912859 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:12.912825    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:12.965715 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:12.965683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:14.129470 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:14.129433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:14.179281 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:14.179248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:15.781079 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:15.781038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 05:59:17.856896 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:17.856865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:17.953127 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:17.953099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.004394 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.004361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.145507 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.145410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.318290 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.318255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.422126 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.422025    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.519038 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.519006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.568564 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.568524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.751025 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.750941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:18.924078 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:18.924026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:19.533325 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:19.533290    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:19.728570 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:19.728538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:20.189425 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:20.189393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:20.553742 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:20.553652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:20.655990 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:20.655955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:20.755283 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:20.755254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:28.243509 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.243481    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 05:59:28.245797 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.245781    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.248343 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.248313    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-whcwh\""
Sep 06 05:59:28.249726 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.249704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 05:59:28.249848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.249775    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 05:59:28.249848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.249784    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 06 05:59:28.249848 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.249807    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 05:59:28.250091 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.249860    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 05:59:28.250091 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.250031    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 05:59:28.250091 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.250055    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 06 05:59:28.255030 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.255008    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 05:59:28.350939 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.350905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.350939 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.350942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vg9\" (UniqueName: \"kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.351143 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.350967    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.351143 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.351054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.351143 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.351094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.351143 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.351134    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451448 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451448 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451674 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451674 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451674 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451543    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vg9\" (UniqueName: \"kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.451674 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.451562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.452213 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.452180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.452332 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.452262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.452332 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.452274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.453962 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.453942    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.454052 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.454040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.459303 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.459281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vg9\" (UniqueName: \"kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9\") pod \"console-547fd4c4bf-xwktm\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") " pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.555178 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.555147    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:28.827038 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.826948    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 05:59:28.830604 ip-10-0-129-33 kubenswrapper[2567]: W0906 05:59:28.830576    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee72fd30_1557_45b3_8617_485c868f0bdb.slice/crio-002f1ad2b75827c15e796f3ea88875e2219be57e59df875dfd4c0056c46d5f7b WatchSource:0}: Error finding container 002f1ad2b75827c15e796f3ea88875e2219be57e59df875dfd4c0056c46d5f7b: Status 404 returned error can't find the container with id 002f1ad2b75827c15e796f3ea88875e2219be57e59df875dfd4c0056c46d5f7b
Sep 06 05:59:28.832479 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.832464    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:59:28.881207 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:28.881179    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547fd4c4bf-xwktm" event={"ID":"ee72fd30-1557-45b3-8617-485c868f0bdb","Type":"ContainerStarted","Data":"002f1ad2b75827c15e796f3ea88875e2219be57e59df875dfd4c0056c46d5f7b"}
Sep 06 05:59:31.892643 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:31.892545    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547fd4c4bf-xwktm" event={"ID":"ee72fd30-1557-45b3-8617-485c868f0bdb","Type":"ContainerStarted","Data":"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"}
Sep 06 05:59:31.910551 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:31.910507    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-547fd4c4bf-xwktm" podStartSLOduration=1.139818612 podStartE2EDuration="3.910492904s" totalImagesPullingTime="2.770674292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:59:31.909097074 +0000 UTC m=+325.612649801" watchObservedRunningTime="2026-09-06 05:59:31.910492904 +0000 UTC m=+325.614045707"
Sep 06 05:59:32.556698 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:32.556656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:32.611560 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:32.611526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:35.632339 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:35.632305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:35.681313 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:35.681278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:36.359780 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:36.359730    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:36.425350 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:36.425318    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:38.555929 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:38.555889    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:38.556398 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:38.555947    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:38.560813 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:38.560790    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:38.918375 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:38.918292    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 05:59:41.157070 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:41.157031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:42.153686 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:42.153649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:42.203428 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:42.203404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:45.219406 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:45.219361    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" podUID="afa089ae-e429-42f8-baad-8abf3dc169c0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 05:59:46.555469 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:46.555425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:46.606627 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:46.606588    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:47.486838 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:47.486805    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 05:59:47.962155 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:47.962119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:48.009167 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:48.009126    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:50.631234 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:50.631203    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:50.685052 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:50.685013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 05:59:55.219664 ip-10-0-129-33 kubenswrapper[2567]: I0906 05:59:55.219623    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" podUID="afa089ae-e429-42f8-baad-8abf3dc169c0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 06:00:05.218816 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.218736    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" podUID="afa089ae-e429-42f8-baad-8abf3dc169c0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 06:00:05.218816 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.218816    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w"
Sep 06 06:00:05.219314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.219284    2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"5bd54f7c0c18325c576a720caf4d1b9b61b75a13d8c830b0679f34c5cc166b8e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 06 06:00:05.219355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.219334    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" podUID="afa089ae-e429-42f8-baad-8abf3dc169c0" containerName="service-proxy" containerID="cri-o://5bd54f7c0c18325c576a720caf4d1b9b61b75a13d8c830b0679f34c5cc166b8e" gracePeriod=30
Sep 06 06:00:05.997847 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.997816    2567 generic.go:356] "Generic (PLEG): container finished" podID="afa089ae-e429-42f8-baad-8abf3dc169c0" containerID="5bd54f7c0c18325c576a720caf4d1b9b61b75a13d8c830b0679f34c5cc166b8e" exitCode=2
Sep 06 06:00:05.998016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.997868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerDied","Data":"5bd54f7c0c18325c576a720caf4d1b9b61b75a13d8c830b0679f34c5cc166b8e"}
Sep 06 06:00:05.998016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:05.997901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d7cfbf956-4ql9w" event={"ID":"afa089ae-e429-42f8-baad-8abf3dc169c0","Type":"ContainerStarted","Data":"2d2dc8880308a8adada196a5cb7d657358de6d1da2e5f1ac244396a4c0cd0ee4"}
Sep 06 06:00:11.187424 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:11.187390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:11.285563 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:11.285530    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:11.637911 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:11.637883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:11.699391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:11.699366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:12.506441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.506401    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-547fd4c4bf-xwktm" podUID="ee72fd30-1557-45b3-8617-485c868f0bdb" containerName="console" containerID="cri-o://44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5" gracePeriod=15
Sep 06 06:00:12.735959 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.735933    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 06:00:12.875185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875160    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.875362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875192    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.875362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875226    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2vg9\" (UniqueName: \"kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.875362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875251    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.875362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875283    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.875362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875321    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert\") pod \"ee72fd30-1557-45b3-8617-485c868f0bdb\" (UID: \"ee72fd30-1557-45b3-8617-485c868f0bdb\") "
Sep 06 06:00:12.876121 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875631    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config" (OuterVolumeSpecName: "console-config") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:00:12.876121 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875709    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:00:12.876121 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.875827    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca" (OuterVolumeSpecName: "service-ca") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:00:12.877658 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.877631    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9" (OuterVolumeSpecName: "kube-api-access-x2vg9") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "kube-api-access-x2vg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:00:12.878000 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.877980    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:00:12.878086 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.878016    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ee72fd30-1557-45b3-8617-485c868f0bdb" (UID: "ee72fd30-1557-45b3-8617-485c868f0bdb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:00:12.976145 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976121    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-oauth-config\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:12.976145 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976142    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee72fd30-1557-45b3-8617-485c868f0bdb-console-serving-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:12.976285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976151    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:12.976285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976160    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-oauth-serving-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:12.976285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976169    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2vg9\" (UniqueName: \"kubernetes.io/projected/ee72fd30-1557-45b3-8617-485c868f0bdb-kube-api-access-x2vg9\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:12.976285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:12.976178    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee72fd30-1557-45b3-8617-485c868f0bdb-console-config\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:00:13.021346 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.021313    2567 generic.go:356] "Generic (PLEG): container finished" podID="ee72fd30-1557-45b3-8617-485c868f0bdb" containerID="44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5" exitCode=0
Sep 06 06:00:13.021485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.021365    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547fd4c4bf-xwktm"
Sep 06 06:00:13.021485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.021405    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547fd4c4bf-xwktm" event={"ID":"ee72fd30-1557-45b3-8617-485c868f0bdb","Type":"ContainerDied","Data":"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"}
Sep 06 06:00:13.021485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.021460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547fd4c4bf-xwktm" event={"ID":"ee72fd30-1557-45b3-8617-485c868f0bdb","Type":"ContainerDied","Data":"002f1ad2b75827c15e796f3ea88875e2219be57e59df875dfd4c0056c46d5f7b"}
Sep 06 06:00:13.021652 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.021486    2567 scope.go:117] "RemoveContainer" containerID="44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"
Sep 06 06:00:13.029591 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.029572    2567 scope.go:117] "RemoveContainer" containerID="44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"
Sep 06 06:00:13.029864 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:00:13.029845    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5\": container with ID starting with 44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5 not found: ID does not exist" containerID="44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"
Sep 06 06:00:13.029919 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.029872    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5"} err="failed to get container status \"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5\": rpc error: code = NotFound desc = could not find container \"44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5\": container with ID starting with 44323f176ce3f331eec4bcc2a4fb860fd046a38658c8c47fbef5e83a96cd6bc5 not found: ID does not exist"
Sep 06 06:00:13.040441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.040420    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 06:00:13.043772 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.043754    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-547fd4c4bf-xwktm"]
Sep 06 06:00:13.740946 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.740899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:13.861229 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:13.861196    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:14.789122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:14.789092    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee72fd30-1557-45b3-8617-485c868f0bdb" path="/var/lib/kubelet/pods/ee72fd30-1557-45b3-8617-485c868f0bdb/volumes"
Sep 06 06:00:20.731216 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:20.731179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 06:00:20.733451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:20.733430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f1efb45-a0eb-4de5-a2d1-1557a281bab6-metrics-certs\") pod \"network-metrics-daemon-pvwwb\" (UID: \"4f1efb45-a0eb-4de5-a2d1-1557a281bab6\") " pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 06:00:20.886461 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:20.886429    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rnw9\""
Sep 06 06:00:20.892858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:20.892838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pvwwb"
Sep 06 06:00:21.159269 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.159242    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pvwwb"]
Sep 06 06:00:21.162089 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:00:21.162048    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1efb45_a0eb_4de5_a2d1_1557a281bab6.slice/crio-66e4fefefac87726f0a23f4a9fe929716535144a47528ece2b7650d3041ca3cd WatchSource:0}: Error finding container 66e4fefefac87726f0a23f4a9fe929716535144a47528ece2b7650d3041ca3cd: Status 404 returned error can't find the container with id 66e4fefefac87726f0a23f4a9fe929716535144a47528ece2b7650d3041ca3cd
Sep 06 06:00:21.708377 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.708343    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:00:21.708673 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.708657    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee72fd30-1557-45b3-8617-485c868f0bdb" containerName="console"
Sep 06 06:00:21.708756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.708677    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee72fd30-1557-45b3-8617-485c868f0bdb" containerName="console"
Sep 06 06:00:21.708756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.708743    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee72fd30-1557-45b3-8617-485c868f0bdb" containerName="console"
Sep 06 06:00:21.710525 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.710501    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.713041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.713016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 06:00:21.714187 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714167    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 06:00:21.714187 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714177    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 06:00:21.714325 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714188    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 06:00:21.714325 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714173    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 06:00:21.714484 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714467    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 06 06:00:21.714587 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714573    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-whcwh\""
Sep 06 06:00:21.714902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.714886    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 06 06:00:21.719432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.718724    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 06 06:00:21.720390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.720318    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:00:21.737727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.738117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.737868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft9gz\" (UniqueName: \"kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.758498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.758457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:21.809592 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.809567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:21.839193 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839238    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft9gz\" (UniqueName: \"kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839560 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839352    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.839560 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.840401 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.839970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.840401 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.840004    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.840401 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.840356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.840621 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.840403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.842439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.842413    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.842534 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.842515    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:21.847583 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:21.847563    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft9gz\" (UniqueName: \"kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz\") pod \"console-657bf96bf8-jbf2l\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") " pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:22.023027 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:22.022932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:22.053089 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:22.053044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pvwwb" event={"ID":"4f1efb45-a0eb-4de5-a2d1-1557a281bab6","Type":"ContainerStarted","Data":"66e4fefefac87726f0a23f4a9fe929716535144a47528ece2b7650d3041ca3cd"}
Sep 06 06:00:22.323286 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:22.323102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:00:22.325724 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:00:22.325697    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d72386_075f_46ba_a7f0_5fcb8586efe3.slice/crio-5d507de4a88884ecd857df37b2120cd6af8fccf162a42f64426a784395d2ff41 WatchSource:0}: Error finding container 5d507de4a88884ecd857df37b2120cd6af8fccf162a42f64426a784395d2ff41: Status 404 returned error can't find the container with id 5d507de4a88884ecd857df37b2120cd6af8fccf162a42f64426a784395d2ff41
Sep 06 06:00:23.058407 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.058367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf96bf8-jbf2l" event={"ID":"43d72386-075f-46ba-a7f0-5fcb8586efe3","Type":"ContainerStarted","Data":"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"}
Sep 06 06:00:23.058867 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.058418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf96bf8-jbf2l" event={"ID":"43d72386-075f-46ba-a7f0-5fcb8586efe3","Type":"ContainerStarted","Data":"5d507de4a88884ecd857df37b2120cd6af8fccf162a42f64426a784395d2ff41"}
Sep 06 06:00:23.060109 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.060088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pvwwb" event={"ID":"4f1efb45-a0eb-4de5-a2d1-1557a281bab6","Type":"ContainerStarted","Data":"83ba186899f4e7403a2b542d34b3ebcc807ff5d194e79ab516fdfd2f08dabbfa"}
Sep 06 06:00:23.060205 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.060125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pvwwb" event={"ID":"4f1efb45-a0eb-4de5-a2d1-1557a281bab6","Type":"ContainerStarted","Data":"e0b66ccde1329ae90fba71d7bd194467dbb52ae3e76301b37adb69d973acfff8"}
Sep 06 06:00:23.075458 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.075420    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-657bf96bf8-jbf2l" podStartSLOduration=2.075406715 podStartE2EDuration="2.075406715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:00:23.073583727 +0000 UTC m=+376.777136462" watchObservedRunningTime="2026-09-06 06:00:23.075406715 +0000 UTC m=+376.778959450"
Sep 06 06:00:23.088160 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.088123    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pvwwb" podStartSLOduration=375.162810741 podStartE2EDuration="6m16.088110161s" totalImagesPullingTime="925.29942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:00:23.087649214 +0000 UTC m=+376.791201953" watchObservedRunningTime="2026-09-06 06:00:23.088110161 +0000 UTC m=+376.791662898"
Sep 06 06:00:23.169945 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.169923    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:23.225820 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:23.225793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:32.023143 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.023107    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:32.023493 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.023169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:32.027930 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.027903    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:32.092550 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.092518    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:00:32.241746 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.241713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:32.298377 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.298272    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:32.379231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.379196    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:32.480183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:32.480151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:42.410661 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:42.410623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:00:42.511588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:00:42.511551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:28.973184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:28.973152    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:29.023737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:29.023649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:40.823568 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:40.823534    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:40.923642 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:40.923610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:50.655024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:50.654991    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:50.749267 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:50.749234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:01:50.801694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:01:50.801663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:16.245112 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:16.245079    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:16.293393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:16.293357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:27.134391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:27.134352    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:02:27.238092 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:27.238037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:27.286388 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:27.286359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:27.338087 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:27.338042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:52.153388 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.153326    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-657bf96bf8-jbf2l" podUID="43d72386-075f-46ba-a7f0-5fcb8586efe3" containerName="console" containerID="cri-o://271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e" gracePeriod=15
Sep 06 06:02:52.386946 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.386924    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:02:52.485777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485679    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.485777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485723    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft9gz\" (UniqueName: \"kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.485777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.485777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.486084 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485794    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.486084 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485810    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.486084 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.485861    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca\") pod \"43d72386-075f-46ba-a7f0-5fcb8586efe3\" (UID: \"43d72386-075f-46ba-a7f0-5fcb8586efe3\") "
Sep 06 06:02:52.486288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.486146    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:02:52.486358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.486291    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:02:52.486358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.486336    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca" (OuterVolumeSpecName: "service-ca") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:02:52.486432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.486409    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config" (OuterVolumeSpecName: "console-config") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:02:52.488012 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.487985    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:02:52.488141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.488047    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:02:52.488141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.488056    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz" (OuterVolumeSpecName: "kube-api-access-ft9gz") pod "43d72386-075f-46ba-a7f0-5fcb8586efe3" (UID: "43d72386-075f-46ba-a7f0-5fcb8586efe3"). InnerVolumeSpecName "kube-api-access-ft9gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:02:52.549104 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.549076    2567 generic.go:356] "Generic (PLEG): container finished" podID="43d72386-075f-46ba-a7f0-5fcb8586efe3" containerID="271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e" exitCode=0
Sep 06 06:02:52.549215 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.549130    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf96bf8-jbf2l"
Sep 06 06:02:52.549215 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.549148    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf96bf8-jbf2l" event={"ID":"43d72386-075f-46ba-a7f0-5fcb8586efe3","Type":"ContainerDied","Data":"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"}
Sep 06 06:02:52.549280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.549221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf96bf8-jbf2l" event={"ID":"43d72386-075f-46ba-a7f0-5fcb8586efe3","Type":"ContainerDied","Data":"5d507de4a88884ecd857df37b2120cd6af8fccf162a42f64426a784395d2ff41"}
Sep 06 06:02:52.549280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.549243    2567 scope.go:117] "RemoveContainer" containerID="271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"
Sep 06 06:02:52.556961 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.556946    2567 scope.go:117] "RemoveContainer" containerID="271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"
Sep 06 06:02:52.557256 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:02:52.557238    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e\": container with ID starting with 271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e not found: ID does not exist" containerID="271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"
Sep 06 06:02:52.557322 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.557264    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e"} err="failed to get container status \"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e\": rpc error: code = NotFound desc = could not find container \"271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e\": container with ID starting with 271c662687c0c6d15730b24c23e57f10aa5a002d1b4b8f7f42ddaaf1252d509e not found: ID does not exist"
Sep 06 06:02:52.570457 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.570436    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:02:52.573657 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.573638    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-657bf96bf8-jbf2l"]
Sep 06 06:02:52.586918 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586902    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586922    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-oauth-serving-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586932    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft9gz\" (UniqueName: \"kubernetes.io/projected/43d72386-075f-46ba-a7f0-5fcb8586efe3-kube-api-access-ft9gz\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586941    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-config\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586949    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43d72386-075f-46ba-a7f0-5fcb8586efe3-trusted-ca-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586957    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-serving-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.586970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.586965    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43d72386-075f-46ba-a7f0-5fcb8586efe3-console-oauth-config\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:02:52.784809 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:52.784736    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d72386-075f-46ba-a7f0-5fcb8586efe3" path="/var/lib/kubelet/pods/43d72386-075f-46ba-a7f0-5fcb8586efe3/volumes"
Sep 06 06:02:56.803010 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:56.802982    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:56.906274 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:56.906234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:02:57.483546 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.483512    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"]
Sep 06 06:02:57.483766 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.483754    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d72386-075f-46ba-a7f0-5fcb8586efe3" containerName="console"
Sep 06 06:02:57.483807 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.483769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d72386-075f-46ba-a7f0-5fcb8586efe3" containerName="console"
Sep 06 06:02:57.483840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.483813    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d72386-075f-46ba-a7f0-5fcb8586efe3" containerName="console"
Sep 06 06:02:57.487837 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.487820    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.490548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.490527    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 06:02:57.490671 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.490528    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 06:02:57.491599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.491580    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sq2v6\""
Sep 06 06:02:57.495393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.495364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"]
Sep 06 06:02:57.623216 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.623178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.623216 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.623219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.623430 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.623278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxlqf\" (UniqueName: \"kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.724412 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.724378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.724599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.724422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxlqf\" (UniqueName: \"kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.724599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.724541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.724784 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.724767    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.724826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.724808    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.733027 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.732996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxlqf\" (UniqueName: \"kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:57.797230 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:57.797144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:02:58.066997 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:58.066964    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"]
Sep 06 06:02:58.070771 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:02:58.070743    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d06ae5_9f52_473d_90e6_8a237763aaf6.slice/crio-0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a WatchSource:0}: Error finding container 0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a: Status 404 returned error can't find the container with id 0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a
Sep 06 06:02:58.571387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:02:58.571347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerStarted","Data":"0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a"}
Sep 06 06:03:03.741655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:03.741622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:04.594008 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:04.593966    2567 generic.go:356] "Generic (PLEG): container finished" podID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerID="238228e44723f919a101933e46f868d5fd9e697fd0d4534137798bbb4c1cda73" exitCode=0
Sep 06 06:03:04.594222 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:04.594053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerDied","Data":"238228e44723f919a101933e46f868d5fd9e697fd0d4534137798bbb4c1cda73"}
Sep 06 06:03:06.603054 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:06.603024    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerStarted","Data":"a9746aef5503cac35c69c67a81465afde4d504142a4ac22674af49fb90a85021"}
Sep 06 06:03:07.608153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:07.608119    2567 generic.go:356] "Generic (PLEG): container finished" podID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerID="a9746aef5503cac35c69c67a81465afde4d504142a4ac22674af49fb90a85021" exitCode=0
Sep 06 06:03:07.608527 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:07.608182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerDied","Data":"a9746aef5503cac35c69c67a81465afde4d504142a4ac22674af49fb90a85021"}
Sep 06 06:03:15.642682 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:15.642642    2567 generic.go:356] "Generic (PLEG): container finished" podID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerID="4fd2a5a6aabca43a920bd444d51d29c36cf325529cf0f5020639248228ff0340" exitCode=0
Sep 06 06:03:15.643048 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:15.642692    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerDied","Data":"4fd2a5a6aabca43a920bd444d51d29c36cf325529cf0f5020639248228ff0340"}
Sep 06 06:03:16.767096 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.767071    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:03:16.871764 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.871731    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util\") pod \"52d06ae5-9f52-473d-90e6-8a237763aaf6\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") "
Sep 06 06:03:16.871936 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.871786    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxlqf\" (UniqueName: \"kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf\") pod \"52d06ae5-9f52-473d-90e6-8a237763aaf6\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") "
Sep 06 06:03:16.871936 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.871826    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle\") pod \"52d06ae5-9f52-473d-90e6-8a237763aaf6\" (UID: \"52d06ae5-9f52-473d-90e6-8a237763aaf6\") "
Sep 06 06:03:16.872519 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.872491    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle" (OuterVolumeSpecName: "bundle") pod "52d06ae5-9f52-473d-90e6-8a237763aaf6" (UID: "52d06ae5-9f52-473d-90e6-8a237763aaf6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:16.874043 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.874020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf" (OuterVolumeSpecName: "kube-api-access-zxlqf") pod "52d06ae5-9f52-473d-90e6-8a237763aaf6" (UID: "52d06ae5-9f52-473d-90e6-8a237763aaf6"). InnerVolumeSpecName "kube-api-access-zxlqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:03:16.876815 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.876792    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util" (OuterVolumeSpecName: "util") pod "52d06ae5-9f52-473d-90e6-8a237763aaf6" (UID: "52d06ae5-9f52-473d-90e6-8a237763aaf6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:16.972590 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.972512    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:16.972590 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.972541    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxlqf\" (UniqueName: \"kubernetes.io/projected/52d06ae5-9f52-473d-90e6-8a237763aaf6-kube-api-access-zxlqf\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:16.972590 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:16.972583    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52d06ae5-9f52-473d-90e6-8a237763aaf6-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:17.651513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:17.651482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq" event={"ID":"52d06ae5-9f52-473d-90e6-8a237763aaf6","Type":"ContainerDied","Data":"0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a"}
Sep 06 06:03:17.651513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:17.651522    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hz6zq"
Sep 06 06:03:17.651764 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:17.651526    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0baaaeef69ec12abc78e350a965b052d9859f28f553f9e625e6860887747b66a"
Sep 06 06:03:23.334673 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:23.334636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:24.715817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:24.715781    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:25.307976 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.307945    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"]
Sep 06 06:03:25.308183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308171    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="pull"
Sep 06 06:03:25.308228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308185    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="pull"
Sep 06 06:03:25.308228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308201    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="util"
Sep 06 06:03:25.308228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308206    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="util"
Sep 06 06:03:25.308228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308215    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="extract"
Sep 06 06:03:25.308228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308221    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="extract"
Sep 06 06:03:25.308374 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.308259    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d06ae5-9f52-473d-90e6-8a237763aaf6" containerName="extract"
Sep 06 06:03:25.311836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.311819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.314897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.314880    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 06 06:03:25.314897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.314889    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c6t4z\""
Sep 06 06:03:25.315042 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.314935    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 06 06:03:25.324465 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.324442    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"]
Sep 06 06:03:25.334727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.334701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlfkz\" (UniqueName: \"kubernetes.io/projected/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-kube-api-access-dlfkz\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.334817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.334743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.435931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.435894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlfkz\" (UniqueName: \"kubernetes.io/projected/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-kube-api-access-dlfkz\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.436118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.435948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.436304 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.436289    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.444522 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.444498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlfkz\" (UniqueName: \"kubernetes.io/projected/1ac0dd0a-610f-427e-8208-7e2a3ba833b8-kube-api-access-dlfkz\") pod \"cert-manager-operator-controller-manager-55c444bf86-x6qtv\" (UID: \"1ac0dd0a-610f-427e-8208-7e2a3ba833b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.620829 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.620739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"
Sep 06 06:03:25.893438 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:25.893408    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv"]
Sep 06 06:03:25.894814 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:03:25.894786    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac0dd0a_610f_427e_8208_7e2a3ba833b8.slice/crio-af2fbbb34f2d14cb364441c84eb48f6dbd8e33676033fc8a27d29a46f917d568 WatchSource:0}: Error finding container af2fbbb34f2d14cb364441c84eb48f6dbd8e33676033fc8a27d29a46f917d568: Status 404 returned error can't find the container with id af2fbbb34f2d14cb364441c84eb48f6dbd8e33676033fc8a27d29a46f917d568
Sep 06 06:03:26.684615 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:26.684577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv" event={"ID":"1ac0dd0a-610f-427e-8208-7e2a3ba833b8","Type":"ContainerStarted","Data":"af2fbbb34f2d14cb364441c84eb48f6dbd8e33676033fc8a27d29a46f917d568"}
Sep 06 06:03:26.951015 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:26.950930    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:27.050838 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:27.050787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:27.692339 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:27.692261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv" event={"ID":"1ac0dd0a-610f-427e-8208-7e2a3ba833b8","Type":"ContainerStarted","Data":"a7609110bee027e6df747976d4891a9cf7a2792feac91d7b512a88e339d33d19"}
Sep 06 06:03:27.714416 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:27.714366    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x6qtv" podStartSLOduration=1.185373161 podStartE2EDuration="2.714347709s" totalImagesPullingTime="1.528974548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:03:27.712975784 +0000 UTC m=+561.416528520" watchObservedRunningTime="2026-09-06 06:03:27.714347709 +0000 UTC m=+561.417900446"
Sep 06 06:03:28.103735 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.103687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:28.216124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.216047    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:28.973378 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.973339    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"]
Sep 06 06:03:28.975608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.975586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:28.978173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.978154    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 06:03:28.978455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.978441    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 06:03:28.978504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.978467    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sq2v6\""
Sep 06 06:03:28.984496 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:28.984476    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"]
Sep 06 06:03:29.059691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.059655    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.059858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.059697    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.059858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.059733    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddhh7\" (UniqueName: \"kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.160118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.160083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddhh7\" (UniqueName: \"kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.160542 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.160201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.160542 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.160250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.160675 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.160659    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.160710 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.160680    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.177322 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.177300    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddhh7\" (UniqueName: \"kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.191901 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.191873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:29.284750 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.284672    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:29.296552 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.296523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:29.556497 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.556475    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"]
Sep 06 06:03:29.558443 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:03:29.558419    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03be5787_1acd_4000_afac_308ea09461ca.slice/crio-40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71 WatchSource:0}: Error finding container 40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71: Status 404 returned error can't find the container with id 40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71
Sep 06 06:03:29.700884 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.700856    2567 generic.go:356] "Generic (PLEG): container finished" podID="03be5787-1acd-4000-afac-308ea09461ca" containerID="1021632b2bb53c8149fa19876f25717bcabd2982711cd46d5f7b2d700ff92fae" exitCode=0
Sep 06 06:03:29.701012 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.700925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh" event={"ID":"03be5787-1acd-4000-afac-308ea09461ca","Type":"ContainerDied","Data":"1021632b2bb53c8149fa19876f25717bcabd2982711cd46d5f7b2d700ff92fae"}
Sep 06 06:03:29.701012 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:29.700962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh" event={"ID":"03be5787-1acd-4000-afac-308ea09461ca","Type":"ContainerStarted","Data":"40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71"}
Sep 06 06:03:31.420461 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:31.420422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:31.611663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:31.611623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:31.710243 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:31.710165    2567 generic.go:356] "Generic (PLEG): container finished" podID="03be5787-1acd-4000-afac-308ea09461ca" containerID="a8145887cd411d1e68fc3b69f428a87fe21e345bea853452d8c3518bcdc240cd" exitCode=0
Sep 06 06:03:31.710362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:31.710237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh" event={"ID":"03be5787-1acd-4000-afac-308ea09461ca","Type":"ContainerDied","Data":"a8145887cd411d1e68fc3b69f428a87fe21e345bea853452d8c3518bcdc240cd"}
Sep 06 06:03:32.715384 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:32.715346    2567 generic.go:356] "Generic (PLEG): container finished" podID="03be5787-1acd-4000-afac-308ea09461ca" containerID="8ddf1004e9d9b9db8bd7c957829f3847b32ffa0085c308da174fe661dea504cd" exitCode=0
Sep 06 06:03:32.715770 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:32.715417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh" event={"ID":"03be5787-1acd-4000-afac-308ea09461ca","Type":"ContainerDied","Data":"8ddf1004e9d9b9db8bd7c957829f3847b32ffa0085c308da174fe661dea504cd"}
Sep 06 06:03:33.837824 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.837800    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:33.893517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.893479    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util\") pod \"03be5787-1acd-4000-afac-308ea09461ca\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") "
Sep 06 06:03:33.893517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.893519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddhh7\" (UniqueName: \"kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7\") pod \"03be5787-1acd-4000-afac-308ea09461ca\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") "
Sep 06 06:03:33.893754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.893553    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle\") pod \"03be5787-1acd-4000-afac-308ea09461ca\" (UID: \"03be5787-1acd-4000-afac-308ea09461ca\") "
Sep 06 06:03:33.893987 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.893963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle" (OuterVolumeSpecName: "bundle") pod "03be5787-1acd-4000-afac-308ea09461ca" (UID: "03be5787-1acd-4000-afac-308ea09461ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:33.895591 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.895564    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7" (OuterVolumeSpecName: "kube-api-access-ddhh7") pod "03be5787-1acd-4000-afac-308ea09461ca" (UID: "03be5787-1acd-4000-afac-308ea09461ca"). InnerVolumeSpecName "kube-api-access-ddhh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:03:33.898784 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.898674    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util" (OuterVolumeSpecName: "util") pod "03be5787-1acd-4000-afac-308ea09461ca" (UID: "03be5787-1acd-4000-afac-308ea09461ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:33.964332 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.964282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:33.994364 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.994296    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:33.994364 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.994325    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddhh7\" (UniqueName: \"kubernetes.io/projected/03be5787-1acd-4000-afac-308ea09461ca-kube-api-access-ddhh7\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:33.994364 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:33.994337    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03be5787-1acd-4000-afac-308ea09461ca-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:34.052195 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:34.052151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:34.724548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:34.724514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh" event={"ID":"03be5787-1acd-4000-afac-308ea09461ca","Type":"ContainerDied","Data":"40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71"}
Sep 06 06:03:34.724720 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:34.724560    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40c3da73163a9d38d0cc51fbde861cc006b363f0143b4133afc327621ad65d71"
Sep 06 06:03:34.724720 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:34.724526    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f56qbh"
Sep 06 06:03:37.564756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:37.564727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:37.662578 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:37.662544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:39.634836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:39.634804    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:42.108516 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:42.108479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:42.208383 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:42.208350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:44.067155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:44.067116    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:44.167317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:44.167286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:46.152354 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152317    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"]
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152531    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="extract"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152541    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="extract"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152553    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="util"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152559    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="util"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152581    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="pull"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152589    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="pull"
Sep 06 06:03:46.152798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.152629    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="03be5787-1acd-4000-afac-308ea09461ca" containerName="extract"
Sep 06 06:03:46.174091 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.174047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"]
Sep 06 06:03:46.174233 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.174159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.176843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.176821    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 06:03:46.177883 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.177865    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sq2v6\""
Sep 06 06:03:46.177955 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.177898    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 06:03:46.280091 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.280025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.280275 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.280141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl8jm\" (UniqueName: \"kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.280275 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.280190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.380839 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.380803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.380989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.380854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl8jm\" (UniqueName: \"kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.380989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.380968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.381269 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.381249    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.381341 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.381325    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.388749 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.388724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl8jm\" (UniqueName: \"kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.482905 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.482810    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:46.751650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.751563    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"]
Sep 06 06:03:46.755303 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:03:46.755277    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7894a4c_dc91_432e_bab1_0f030dce82a1.slice/crio-a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2 WatchSource:0}: Error finding container a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2: Status 404 returned error can't find the container with id a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2
Sep 06 06:03:46.769284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:46.769251    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw" event={"ID":"c7894a4c-dc91-432e-bab1-0f030dce82a1","Type":"ContainerStarted","Data":"a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2"}
Sep 06 06:03:47.049381 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:47.049299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:47.773942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:47.773856    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerID="b94d08434cec6e0cfbb2e442930ba35af2e2b8d3ca08c56da36efe927596c2ea" exitCode=0
Sep 06 06:03:47.773942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:47.773928    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw" event={"ID":"c7894a4c-dc91-432e-bab1-0f030dce82a1","Type":"ContainerDied","Data":"b94d08434cec6e0cfbb2e442930ba35af2e2b8d3ca08c56da36efe927596c2ea"}
Sep 06 06:03:48.779556 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:48.779520    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerID="858bb96f28647b37965fe57d50bd06903be655a28d27a6c000865dfffebcaeb4" exitCode=0
Sep 06 06:03:48.779986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:48.779574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw" event={"ID":"c7894a4c-dc91-432e-bab1-0f030dce82a1","Type":"ContainerDied","Data":"858bb96f28647b37965fe57d50bd06903be655a28d27a6c000865dfffebcaeb4"}
Sep 06 06:03:49.785968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.785935    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerID="6db429cabd78c8690134c9e59180b08abf76610407367443ec3096c3f93904c7" exitCode=0
Sep 06 06:03:49.786351 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.785998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw" event={"ID":"c7894a4c-dc91-432e-bab1-0f030dce82a1","Type":"ContainerDied","Data":"6db429cabd78c8690134c9e59180b08abf76610407367443ec3096c3f93904c7"}
Sep 06 06:03:49.838776 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.838744    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5hvtj"]
Sep 06 06:03:49.840876 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.840860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:49.843419 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.843401    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 06 06:03:49.843510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.843415    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 06 06:03:49.843562 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.843538    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bht6m\""
Sep 06 06:03:49.849585 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.849563    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5hvtj"]
Sep 06 06:03:49.907727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.907698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-bound-sa-token\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:49.907727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:49.907726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vlp6\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-kube-api-access-8vlp6\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.009046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.009008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-bound-sa-token\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.009046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.009048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vlp6\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-kube-api-access-8vlp6\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.017760 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.017725    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-bound-sa-token\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.017948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.017930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vlp6\" (UniqueName: \"kubernetes.io/projected/d6bf54f6-be59-482d-a64a-a73fac6205bc-kube-api-access-8vlp6\") pod \"cert-manager-55b4d954d5-5hvtj\" (UID: \"d6bf54f6-be59-482d-a64a-a73fac6205bc\") " pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.149719 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.149691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5hvtj"
Sep 06 06:03:50.415933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.415858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5hvtj"]
Sep 06 06:03:50.418942 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:03:50.418911    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6bf54f6_be59_482d_a64a_a73fac6205bc.slice/crio-41c6f048c72599d0f0f699153474d866bfb3cf367789a1b5ca4d083cf86fdf67 WatchSource:0}: Error finding container 41c6f048c72599d0f0f699153474d866bfb3cf367789a1b5ca4d083cf86fdf67: Status 404 returned error can't find the container with id 41c6f048c72599d0f0f699153474d866bfb3cf367789a1b5ca4d083cf86fdf67
Sep 06 06:03:50.791505 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.791425    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5hvtj" event={"ID":"d6bf54f6-be59-482d-a64a-a73fac6205bc","Type":"ContainerStarted","Data":"41c6f048c72599d0f0f699153474d866bfb3cf367789a1b5ca4d083cf86fdf67"}
Sep 06 06:03:50.899038 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:50.899014    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:51.016041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.016011    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle\") pod \"c7894a4c-dc91-432e-bab1-0f030dce82a1\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") "
Sep 06 06:03:51.016206 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.016052    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl8jm\" (UniqueName: \"kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm\") pod \"c7894a4c-dc91-432e-bab1-0f030dce82a1\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") "
Sep 06 06:03:51.016206 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.016095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util\") pod \"c7894a4c-dc91-432e-bab1-0f030dce82a1\" (UID: \"c7894a4c-dc91-432e-bab1-0f030dce82a1\") "
Sep 06 06:03:51.016744 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.016706    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle" (OuterVolumeSpecName: "bundle") pod "c7894a4c-dc91-432e-bab1-0f030dce82a1" (UID: "c7894a4c-dc91-432e-bab1-0f030dce82a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:51.018093 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.018071    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm" (OuterVolumeSpecName: "kube-api-access-jl8jm") pod "c7894a4c-dc91-432e-bab1-0f030dce82a1" (UID: "c7894a4c-dc91-432e-bab1-0f030dce82a1"). InnerVolumeSpecName "kube-api-access-jl8jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:03:51.021589 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.021559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util" (OuterVolumeSpecName: "util") pod "c7894a4c-dc91-432e-bab1-0f030dce82a1" (UID: "c7894a4c-dc91-432e-bab1-0f030dce82a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:03:51.116917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.116886    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:51.116917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.116913    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl8jm\" (UniqueName: \"kubernetes.io/projected/c7894a4c-dc91-432e-bab1-0f030dce82a1-kube-api-access-jl8jm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:51.116917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.116925    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7894a4c-dc91-432e-bab1-0f030dce82a1-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:03:51.799191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.799150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw" event={"ID":"c7894a4c-dc91-432e-bab1-0f030dce82a1","Type":"ContainerDied","Data":"a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2"}
Sep 06 06:03:51.799191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.799184    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjpvcw"
Sep 06 06:03:51.799668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:51.799207    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4de8fa017ccc644173034d053e60728c9b674e42ce5a7ecf7dadea43c14a4e2"
Sep 06 06:03:53.808621 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:53.808580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5hvtj" event={"ID":"d6bf54f6-be59-482d-a64a-a73fac6205bc","Type":"ContainerStarted","Data":"2b4268062cc8af6b2750d2f1265475dfa802b1151973dd50269a02a261056512"}
Sep 06 06:03:53.823377 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:53.823331    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5hvtj" podStartSLOduration=2.177107689 podStartE2EDuration="4.823316283s" totalImagesPullingTime="2.646208594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:03:53.822183558 +0000 UTC m=+587.525736293" watchObservedRunningTime="2026-09-06 06:03:53.823316283 +0000 UTC m=+587.526869018"
Sep 06 06:03:57.266081 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:57.266024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:03:57.338348 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:03:57.338311    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:03.010773 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.010733    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"]
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.010982    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="util"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.010992    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="util"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.011000    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="extract"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.011006    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="extract"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.011014    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="pull"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.011020    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="pull"
Sep 06 06:04:03.011164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.011077    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7894a4c-dc91-432e-bab1-0f030dce82a1" containerName="extract"
Sep 06 06:04:03.015355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.015336    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.018008 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.017980    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 06:04:03.018008 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.018005    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 06:04:03.018185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.018126    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sq2v6\""
Sep 06 06:04:03.023942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.023920    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"]
Sep 06 06:04:03.100828 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.100793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.100976 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.100834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pc2d\" (UniqueName: \"kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.100976 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.100854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.160403 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.160367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:03.201973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.201950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.202097 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.201983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pc2d\" (UniqueName: \"kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.202097 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.202002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.202326 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.202311    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.202386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.202369    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.221556 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.221535    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pc2d\" (UniqueName: \"kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.324679 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.324646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:03.592522 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.592477    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"]
Sep 06 06:04:03.594617 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:03.594584    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d61403_48c4_431d_984e_a5db9d0c2854.slice/crio-a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468 WatchSource:0}: Error finding container a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468: Status 404 returned error can't find the container with id a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468
Sep 06 06:04:03.848569 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.848481    2567 generic.go:356] "Generic (PLEG): container finished" podID="60d61403-48c4-431d-984e-a5db9d0c2854" containerID="12402a0b1d2dfc506155314e3ab80e34f5597ee63d8b84fd6dd622bfb0fbcc9e" exitCode=0
Sep 06 06:04:03.848569 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.848519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn" event={"ID":"60d61403-48c4-431d-984e-a5db9d0c2854","Type":"ContainerDied","Data":"12402a0b1d2dfc506155314e3ab80e34f5597ee63d8b84fd6dd622bfb0fbcc9e"}
Sep 06 06:04:03.848569 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:03.848567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn" event={"ID":"60d61403-48c4-431d-984e-a5db9d0c2854","Type":"ContainerStarted","Data":"a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468"}
Sep 06 06:04:04.119419 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.119338    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"]
Sep 06 06:04:04.121480 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.121464    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.126661 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.126639    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cvg9l\""
Sep 06 06:04:04.126811 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.126695    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 06:04:04.126925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.126850    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 06:04:04.127003 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.126962    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 06:04:04.127132 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.127114    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 06:04:04.138312 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.138286    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"]
Sep 06 06:04:04.211142 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.211111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-tmp\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.211316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.211157    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-manifests\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.211316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.211189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.211316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.211231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lxtw\" (UniqueName: \"kubernetes.io/projected/56df9ec4-77bc-491e-b774-0da95d7c4a5a-kube-api-access-4lxtw\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.211316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.211271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.311962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.311915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.311962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.311964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lxtw\" (UniqueName: \"kubernetes.io/projected/56df9ec4-77bc-491e-b774-0da95d7c4a5a-kube-api-access-4lxtw\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.312227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.312007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.312227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.312036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-tmp\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.312227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.312107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-manifests\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.312501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.312480    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-manifests\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.314359 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.314335    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56df9ec4-77bc-491e-b774-0da95d7c4a5a-tmp\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.314586 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.314570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.314650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.314616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56df9ec4-77bc-491e-b774-0da95d7c4a5a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.319806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.319782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lxtw\" (UniqueName: \"kubernetes.io/projected/56df9ec4-77bc-491e-b774-0da95d7c4a5a-kube-api-access-4lxtw\") pod \"opendatahub-operator-controller-manager-b897cd959-l9k4j\" (UID: \"56df9ec4-77bc-491e-b774-0da95d7c4a5a\") " pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.402768 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.402689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:04.431656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.431625    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:04.511225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.511188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:04.626976 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.626944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:04.707352 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.707276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:04.731439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.731408    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"]
Sep 06 06:04:04.733577 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:04.733554    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56df9ec4_77bc_491e_b774_0da95d7c4a5a.slice/crio-18bd227a7551c5acd59907a9dc42db83fcf58eb73f926aa94c66c078a8e51dda WatchSource:0}: Error finding container 18bd227a7551c5acd59907a9dc42db83fcf58eb73f926aa94c66c078a8e51dda: Status 404 returned error can't find the container with id 18bd227a7551c5acd59907a9dc42db83fcf58eb73f926aa94c66c078a8e51dda
Sep 06 06:04:04.860015 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.859979    2567 generic.go:356] "Generic (PLEG): container finished" podID="60d61403-48c4-431d-984e-a5db9d0c2854" containerID="2c2cf8d549b24d559034df79e73b5ead04c4a3020099c3a660e6ca2843e091fe" exitCode=0
Sep 06 06:04:04.860189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.860079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn" event={"ID":"60d61403-48c4-431d-984e-a5db9d0c2854","Type":"ContainerDied","Data":"2c2cf8d549b24d559034df79e73b5ead04c4a3020099c3a660e6ca2843e091fe"}
Sep 06 06:04:04.861361 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:04.861331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j" event={"ID":"56df9ec4-77bc-491e-b774-0da95d7c4a5a","Type":"ContainerStarted","Data":"18bd227a7551c5acd59907a9dc42db83fcf58eb73f926aa94c66c078a8e51dda"}
Sep 06 06:04:05.868807 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:05.868766    2567 generic.go:356] "Generic (PLEG): container finished" podID="60d61403-48c4-431d-984e-a5db9d0c2854" containerID="cbd08221dcf1d009b9ef38889924f69f572daac62f6abbdf503e4525f3785c29" exitCode=0
Sep 06 06:04:05.869203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:05.868870    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn" event={"ID":"60d61403-48c4-431d-984e-a5db9d0c2854","Type":"ContainerDied","Data":"cbd08221dcf1d009b9ef38889924f69f572daac62f6abbdf503e4525f3785c29"}
Sep 06 06:04:07.606673 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.606648    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:07.741547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.741512    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle\") pod \"60d61403-48c4-431d-984e-a5db9d0c2854\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") "
Sep 06 06:04:07.741547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.741554    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util\") pod \"60d61403-48c4-431d-984e-a5db9d0c2854\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") "
Sep 06 06:04:07.741740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.741620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pc2d\" (UniqueName: \"kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d\") pod \"60d61403-48c4-431d-984e-a5db9d0c2854\" (UID: \"60d61403-48c4-431d-984e-a5db9d0c2854\") "
Sep 06 06:04:07.742214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.742187    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle" (OuterVolumeSpecName: "bundle") pod "60d61403-48c4-431d-984e-a5db9d0c2854" (UID: "60d61403-48c4-431d-984e-a5db9d0c2854"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:04:07.743627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.743595    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d" (OuterVolumeSpecName: "kube-api-access-8pc2d") pod "60d61403-48c4-431d-984e-a5db9d0c2854" (UID: "60d61403-48c4-431d-984e-a5db9d0c2854"). InnerVolumeSpecName "kube-api-access-8pc2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:04:07.746422 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.746392    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util" (OuterVolumeSpecName: "util") pod "60d61403-48c4-431d-984e-a5db9d0c2854" (UID: "60d61403-48c4-431d-984e-a5db9d0c2854"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:04:07.842153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.842048    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:07.842153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.842101    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60d61403-48c4-431d-984e-a5db9d0c2854-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:07.842153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.842111    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pc2d\" (UniqueName: \"kubernetes.io/projected/60d61403-48c4-431d-984e-a5db9d0c2854-kube-api-access-8pc2d\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:07.879821 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.879789    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn"
Sep 06 06:04:07.879943 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.879791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06whmn" event={"ID":"60d61403-48c4-431d-984e-a5db9d0c2854","Type":"ContainerDied","Data":"a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468"}
Sep 06 06:04:07.879943 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.879916    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a68409873c583df6704901aadbffe8c51ff52b21b2e41bc3410d8dd1db1e5468"
Sep 06 06:04:07.881460 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:07.881435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j" event={"ID":"56df9ec4-77bc-491e-b774-0da95d7c4a5a","Type":"ContainerStarted","Data":"60f0157162eeda73d83247eaca24f7718fea1358222f924ad325445c98fbdf23"}
Sep 06 06:04:08.887110 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:08.887079    2567 generic.go:356] "Generic (PLEG): container finished" podID="56df9ec4-77bc-491e-b774-0da95d7c4a5a" containerID="60f0157162eeda73d83247eaca24f7718fea1358222f924ad325445c98fbdf23" exitCode=0
Sep 06 06:04:08.887491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:08.887168    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j" event={"ID":"56df9ec4-77bc-491e-b774-0da95d7c4a5a","Type":"ContainerDied","Data":"60f0157162eeda73d83247eaca24f7718fea1358222f924ad325445c98fbdf23"}
Sep 06 06:04:09.893935 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:09.893902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j" event={"ID":"56df9ec4-77bc-491e-b774-0da95d7c4a5a","Type":"ContainerStarted","Data":"7a229b1eb9619ab5b2e1f48613284398b5d44d889aed470f8c8072a3142c473f"}
Sep 06 06:04:09.896108 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:09.894015    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:09.916454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:09.916408    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j" podStartSLOduration=1.6301864959999999 podStartE2EDuration="5.916393669s" totalImagesPullingTime="3.258820746s" totalInitContainerRuntime="1.027386427s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:09.914810322 +0000 UTC m=+603.618363058" watchObservedRunningTime="2026-09-06 06:04:09.916393669 +0000 UTC m=+603.619946404"
Sep 06 06:04:16.362474 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:16.362440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:16.464432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:16.464400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:20.900203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:20.900170    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b897cd959-l9k4j"
Sep 06 06:04:22.421605 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:22.421571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:22.974199 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:22.974167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:23.093143 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.093101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:23.266714 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266633    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"]
Sep 06 06:04:23.266885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266873    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="util"
Sep 06 06:04:23.266931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266887    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="util"
Sep 06 06:04:23.266931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266897    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="pull"
Sep 06 06:04:23.266931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266902    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="pull"
Sep 06 06:04:23.266931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266917    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="extract"
Sep 06 06:04:23.266931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266923    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="extract"
Sep 06 06:04:23.267127 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.266965    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d61403-48c4-431d-984e-a5db9d0c2854" containerName="extract"
Sep 06 06:04:23.271490 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.271469    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.272989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.272961    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:23.276776 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.276753    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 06 06:04:23.276869 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.276850    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 06:04:23.277648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.277628    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 06 06:04:23.278755 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.278736    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 06:04:23.302699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.302667    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"]
Sep 06 06:04:23.362299 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.362270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/addb6816-ed73-4db1-a01e-2c4b72b0b488-tmp\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.362456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.362317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brlk\" (UniqueName: \"kubernetes.io/projected/addb6816-ed73-4db1-a01e-2c4b72b0b488-kube-api-access-4brlk\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.362456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.362350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/addb6816-ed73-4db1-a01e-2c4b72b0b488-tls-certs\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.462923 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.462880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/addb6816-ed73-4db1-a01e-2c4b72b0b488-tls-certs\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.463421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.462948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/addb6816-ed73-4db1-a01e-2c4b72b0b488-tmp\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.463421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.463007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brlk\" (UniqueName: \"kubernetes.io/projected/addb6816-ed73-4db1-a01e-2c4b72b0b488-kube-api-access-4brlk\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.465251 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.465225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/addb6816-ed73-4db1-a01e-2c4b72b0b488-tmp\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.465435 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.465416    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/addb6816-ed73-4db1-a01e-2c4b72b0b488-tls-certs\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.481292 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.481249    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brlk\" (UniqueName: \"kubernetes.io/projected/addb6816-ed73-4db1-a01e-2c4b72b0b488-kube-api-access-4brlk\") pod \"kube-auth-proxy-696f56895b-6rlkm\" (UID: \"addb6816-ed73-4db1-a01e-2c4b72b0b488\") " pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.528219 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.528111    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:23.580878 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.580841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"
Sep 06 06:04:23.858329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.858305    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-696f56895b-6rlkm"]
Sep 06 06:04:23.860307 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:23.860271    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddb6816_ed73_4db1_a01e_2c4b72b0b488.slice/crio-97be482b3f7ab2f2c3a74b74e25a79bfd3dadff556005c87d1431da287c5b726 WatchSource:0}: Error finding container 97be482b3f7ab2f2c3a74b74e25a79bfd3dadff556005c87d1431da287c5b726: Status 404 returned error can't find the container with id 97be482b3f7ab2f2c3a74b74e25a79bfd3dadff556005c87d1431da287c5b726
Sep 06 06:04:23.950460 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:23.950429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm" event={"ID":"addb6816-ed73-4db1-a01e-2c4b72b0b488","Type":"ContainerStarted","Data":"97be482b3f7ab2f2c3a74b74e25a79bfd3dadff556005c87d1431da287c5b726"}
Sep 06 06:04:24.600667 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.600595    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:24.602529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.602499    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:24.603090 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.603072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.605776 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.605752    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 06 06:04:24.605879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.605773    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 06 06:04:24.605879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.605757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xg9sk\""
Sep 06 06:04:24.605879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.605762    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 06:04:24.605879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.605873    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 06 06:04:24.614800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.614772    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:24.666702 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.666662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:24.773382 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773559 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9hph\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773559 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773559 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773583    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.773687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.773672    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.874879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.874797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.874879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.874855    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9hph\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875122    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.875770 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.875728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.877935 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.877889    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.877935 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.877928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.878137 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.877938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.878137 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.878013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.888617 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.888592    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.894471 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.894447    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9hph\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph\") pod \"istiod-openshift-gateway-7b7995d5cd-4tg8b\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:24.914258 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:24.914239    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:25.084556 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:25.084514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:25.147986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:25.147897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:25.203703 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:25.203664    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:25.207555 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:25.207520    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb895c0ef_76a8_4288_b934_0f893dd81279.slice/crio-ab8c3fe955322913ba3cc95039972cd08bd06d97fa6975c9f8154b84159b3f70 WatchSource:0}: Error finding container ab8c3fe955322913ba3cc95039972cd08bd06d97fa6975c9f8154b84159b3f70: Status 404 returned error can't find the container with id ab8c3fe955322913ba3cc95039972cd08bd06d97fa6975c9f8154b84159b3f70
Sep 06 06:04:25.542739 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:25.542639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:25.961108 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:25.961054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" event={"ID":"b895c0ef-76a8-4288-b934-0f893dd81279","Type":"ContainerStarted","Data":"ab8c3fe955322913ba3cc95039972cd08bd06d97fa6975c9f8154b84159b3f70"}
Sep 06 06:04:26.966886 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:26.966797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm" event={"ID":"addb6816-ed73-4db1-a01e-2c4b72b0b488","Type":"ContainerStarted","Data":"eea2a0e290667459ca813f2df5115e31a24eb27709b6c9034506bd3d2a3d3477"}
Sep 06 06:04:29.416948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.416877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:29.486556 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.486516    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:04:29.486684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.486589    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:04:29.517027 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.516984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:29.617284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.617250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:29.962504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.962471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:29.984049 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.984018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" event={"ID":"b895c0ef-76a8-4288-b934-0f893dd81279","Type":"ContainerStarted","Data":"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"}
Sep 06 06:04:29.984231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:29.984200    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:30.005410 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.005361    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-696f56895b-6rlkm" podStartSLOduration=4.219696689 podStartE2EDuration="7.005345587s" totalImagesPullingTime="2.785648898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:26.982697589 +0000 UTC m=+620.686250324" watchObservedRunningTime="2026-09-06 06:04:30.005345587 +0000 UTC m=+623.708898322"
Sep 06 06:04:30.006241 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.006212    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" podStartSLOduration=1.730192642 podStartE2EDuration="6.0062022s" totalImagesPullingTime="4.276009558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:30.003696756 +0000 UTC m=+623.707249491" watchObservedRunningTime="2026-09-06 06:04:30.0062022 +0000 UTC m=+623.709754936"
Sep 06 06:04:30.062046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.062014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:30.351191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.351145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:30.444371 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.444321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:30.893424 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.893390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:30.989609 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:30.989580    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:31.061368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:31.061337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:31.161558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:31.161472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.104704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.104659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.202686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.202647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.305123 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.305094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.597002 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.596966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.610789 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.610759    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p9tsq"]
Sep 06 06:04:34.615006 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.614985    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.617570 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.617553    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 06 06:04:34.617668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.617609    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-p46mw\""
Sep 06 06:04:34.622978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.622950    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p9tsq"]
Sep 06 06:04:34.755740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.755708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebbd126-638a-46b5-9cbf-44ba12db945c-tls-certs\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.755740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.755746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjjls\" (UniqueName: \"kubernetes.io/projected/1ebbd126-638a-46b5-9cbf-44ba12db945c-kube-api-access-pjjls\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.857318 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.857223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebbd126-638a-46b5-9cbf-44ba12db945c-tls-certs\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.857318 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.857261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjls\" (UniqueName: \"kubernetes.io/projected/1ebbd126-638a-46b5-9cbf-44ba12db945c-kube-api-access-pjjls\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.859926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.859889    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebbd126-638a-46b5-9cbf-44ba12db945c-tls-certs\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.866411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.866381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjls\" (UniqueName: \"kubernetes.io/projected/1ebbd126-638a-46b5-9cbf-44ba12db945c-kube-api-access-pjjls\") pod \"model-serving-api-54d89bd5fc-p9tsq\" (UID: \"1ebbd126-638a-46b5-9cbf-44ba12db945c\") " pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:34.878136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.878111    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:34.926144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:34.926115    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:35.199091 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:35.199051    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p9tsq"]
Sep 06 06:04:35.201573 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:35.201541    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebbd126_638a_46b5_9cbf_44ba12db945c.slice/crio-9d5ecdc295526bdd573b948c65e85771eea7791436a342f0bc6ee1bf6ad606f9 WatchSource:0}: Error finding container 9d5ecdc295526bdd573b948c65e85771eea7791436a342f0bc6ee1bf6ad606f9: Status 404 returned error can't find the container with id 9d5ecdc295526bdd573b948c65e85771eea7791436a342f0bc6ee1bf6ad606f9
Sep 06 06:04:35.204118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:35.204094    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 06:04:36.010845 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:36.010797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq" event={"ID":"1ebbd126-638a-46b5-9cbf-44ba12db945c","Type":"ContainerStarted","Data":"9d5ecdc295526bdd573b948c65e85771eea7791436a342f0bc6ee1bf6ad606f9"}
Sep 06 06:04:37.021026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:37.020925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq" event={"ID":"1ebbd126-638a-46b5-9cbf-44ba12db945c","Type":"ContainerStarted","Data":"e792c658ce9c7660b467417d6a3e3f3401e80f6d4b2a8eff85c886c14b27d399"}
Sep 06 06:04:37.021026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:37.021019    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:04:37.037647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:37.037596    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq" podStartSLOduration=1.609175493 podStartE2EDuration="3.037580406s" totalImagesPullingTime="1.428404913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:37.035464634 +0000 UTC m=+630.739017369" watchObservedRunningTime="2026-09-06 06:04:37.037580406 +0000 UTC m=+630.741133149"
Sep 06 06:04:40.889956 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:40.889925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:40.997081 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:40.997030    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:41.058470 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.058429    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"]
Sep 06 06:04:41.064591 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.064569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.072214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.072191    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"]
Sep 06 06:04:41.210183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shj44\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-kube-api-access-shj44\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210292    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.210386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.210347    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shj44\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-kube-api-access-shj44\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.311826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.311792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.312536 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.312509    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.314311 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.314287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.314434 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.314354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.314434 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.314375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.314633 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.314614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.320423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.320390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shj44\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-kube-api-access-shj44\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.320617 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.320597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94ccf2fc-7a70-46d8-b1e9-606aebe798d9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d2jb4\" (UID: \"94ccf2fc-7a70-46d8-b1e9-606aebe798d9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.373670 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.373638    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:41.646811 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.646788    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"]
Sep 06 06:04:41.648443 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:41.648415    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ccf2fc_7a70_46d8_b1e9_606aebe798d9.slice/crio-32e9de345ad769594edf6129137958fbc120c781d5f7bcc58786a4b4dd4876b5 WatchSource:0}: Error finding container 32e9de345ad769594edf6129137958fbc120c781d5f7bcc58786a4b4dd4876b5: Status 404 returned error can't find the container with id 32e9de345ad769594edf6129137958fbc120c781d5f7bcc58786a4b4dd4876b5
Sep 06 06:04:41.650533 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.650494    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:04:41.650624 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:41.650570    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:04:42.044198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.044112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4" event={"ID":"94ccf2fc-7a70-46d8-b1e9-606aebe798d9","Type":"ContainerStarted","Data":"c64e5a49376c9658dffca5129d707757e282e08681ae3685de5b9ec924b176ca"}
Sep 06 06:04:42.044198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.044163    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4" event={"ID":"94ccf2fc-7a70-46d8-b1e9-606aebe798d9","Type":"ContainerStarted","Data":"32e9de345ad769594edf6129137958fbc120c781d5f7bcc58786a4b4dd4876b5"}
Sep 06 06:04:42.044644 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.044301    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:42.046035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.046010    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-d2jb4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 06:04:42.046148 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.046054    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4" podUID="94ccf2fc-7a70-46d8-b1e9-606aebe798d9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 06:04:42.098288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:42.098243    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4" podStartSLOduration=1.098226899 podStartE2EDuration="1.098226899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:42.096749725 +0000 UTC m=+635.800302461" watchObservedRunningTime="2026-09-06 06:04:42.098226899 +0000 UTC m=+635.801779635"
Sep 06 06:04:43.049173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.049142    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d2jb4"
Sep 06 06:04:43.129708 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.129678    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:43.129875 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.129855    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" podUID="b895c0ef-76a8-4288-b934-0f893dd81279" containerName="discovery" containerID="cri-o://fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a" gracePeriod=30
Sep 06 06:04:43.372533 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.372509    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:43.427893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.427863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.427893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.427895    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.427911    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.427968    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9hph\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.427997    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.428050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.428107    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap\") pod \"b895c0ef-76a8-4288-b934-0f893dd81279\" (UID: \"b895c0ef-76a8-4288-b934-0f893dd81279\") "
Sep 06 06:04:43.428704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.428668    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:04:43.430755 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.430726    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:04:43.430876 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.430753    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:04:43.430876 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.430791    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs" (OuterVolumeSpecName: "local-certs") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:04:43.430876 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.430869    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts" (OuterVolumeSpecName: "cacerts") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:04:43.431041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.431023    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph" (OuterVolumeSpecName: "kube-api-access-c9hph") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "kube-api-access-c9hph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:04:43.431675 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.431656    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token" (OuterVolumeSpecName: "istio-token") pod "b895c0ef-76a8-4288-b934-0f893dd81279" (UID: "b895c0ef-76a8-4288-b934-0f893dd81279"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:04:43.486939 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.486893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:43.528737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528710    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-ca-configmap\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528735    2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-kubeconfig\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528745    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-istio-token\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528753    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-istio-csr-dns-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528762    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9hph\" (UniqueName: \"kubernetes.io/projected/b895c0ef-76a8-4288-b934-0f893dd81279-kube-api-access-c9hph\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528770    2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b895c0ef-76a8-4288-b934-0f893dd81279-cacerts\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.528965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.528778    2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b895c0ef-76a8-4288-b934-0f893dd81279-local-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:04:43.610368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.610269    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"]
Sep 06 06:04:43.610606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.610591    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b895c0ef-76a8-4288-b934-0f893dd81279" containerName="discovery"
Sep 06 06:04:43.610652 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.610608    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b895c0ef-76a8-4288-b934-0f893dd81279" containerName="discovery"
Sep 06 06:04:43.610699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.610661    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b895c0ef-76a8-4288-b934-0f893dd81279" containerName="discovery"
Sep 06 06:04:43.612893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.612871    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.616887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.616865    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2db7n\""
Sep 06 06:04:43.617088 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.617073    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 06 06:04:43.626781 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.626747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:43.629822 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.629801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jg2t\" (UniqueName: \"kubernetes.io/projected/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-kube-api-access-9jg2t\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.629907 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.629835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-cert\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.639980 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.639945    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"]
Sep 06 06:04:43.730184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.730146    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:43.730387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.730261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jg2t\" (UniqueName: \"kubernetes.io/projected/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-kube-api-access-9jg2t\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.730387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.730290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-cert\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.732729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.732703    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-cert\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.739406 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.739384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jg2t\" (UniqueName: \"kubernetes.io/projected/94ff23f0-bb45-48aa-a7bd-4132cbfd0e81-kube-api-access-9jg2t\") pod \"kserve-controller-manager-d6f4b65b8-wpf4d\" (UID: \"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:43.922996 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:43.922908    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:44.053441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.053411    2567 generic.go:356] "Generic (PLEG): container finished" podID="b895c0ef-76a8-4288-b934-0f893dd81279" containerID="fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a" exitCode=0
Sep 06 06:04:44.053887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.053465    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"
Sep 06 06:04:44.053887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.053494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" event={"ID":"b895c0ef-76a8-4288-b934-0f893dd81279","Type":"ContainerDied","Data":"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"}
Sep 06 06:04:44.053887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.053541    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b" event={"ID":"b895c0ef-76a8-4288-b934-0f893dd81279","Type":"ContainerDied","Data":"ab8c3fe955322913ba3cc95039972cd08bd06d97fa6975c9f8154b84159b3f70"}
Sep 06 06:04:44.053887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.053563    2567 scope.go:117] "RemoveContainer" containerID="fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"
Sep 06 06:04:44.061567 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.061546    2567 scope.go:117] "RemoveContainer" containerID="fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"
Sep 06 06:04:44.061842 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:04:44.061823    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a\": container with ID starting with fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a not found: ID does not exist" containerID="fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"
Sep 06 06:04:44.061918 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.061849    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a"} err="failed to get container status \"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a\": rpc error: code = NotFound desc = could not find container \"fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a\": container with ID starting with fd5888d96c3e7752b1330884413569a0bf1bbd7455350d4e4a81b045c680b16a not found: ID does not exist"
Sep 06 06:04:44.075246 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.075226    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:44.079474 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.079455    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4tg8b"]
Sep 06 06:04:44.191284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.191259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"]
Sep 06 06:04:44.193119 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:04:44.193096    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ff23f0_bb45_48aa_a7bd_4132cbfd0e81.slice/crio-5500e47c6f4acadd44900e9404289b5dbe06fb752ed1315c2f260d6fea699963 WatchSource:0}: Error finding container 5500e47c6f4acadd44900e9404289b5dbe06fb752ed1315c2f260d6fea699963: Status 404 returned error can't find the container with id 5500e47c6f4acadd44900e9404289b5dbe06fb752ed1315c2f260d6fea699963
Sep 06 06:04:44.786555 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.786518    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b895c0ef-76a8-4288-b934-0f893dd81279" path="/var/lib/kubelet/pods/b895c0ef-76a8-4288-b934-0f893dd81279/volumes"
Sep 06 06:04:44.867597 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.867545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:44.962238 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:44.962198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:45.060371 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:45.060342    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d" event={"ID":"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81","Type":"ContainerStarted","Data":"5500e47c6f4acadd44900e9404289b5dbe06fb752ed1315c2f260d6fea699963"}
Sep 06 06:04:45.076599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:45.076576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:45.842536 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:45.842493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:46.315881 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:46.315840    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:04:47.070939 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:47.070903    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d" event={"ID":"94ff23f0-bb45-48aa-a7bd-4132cbfd0e81","Type":"ContainerStarted","Data":"e4c3c4ea904aa635b02f3529c17a0cf753e6c033fc11f3f50381fd488cf02fa4"}
Sep 06 06:04:47.071136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:47.070973    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:04:47.087036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:47.086977    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d" podStartSLOduration=1.784828198 podStartE2EDuration="4.086958269s" totalImagesPullingTime="2.302130071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:04:47.086437299 +0000 UTC m=+640.789990037" watchObservedRunningTime="2026-09-06 06:04:47.086958269 +0000 UTC m=+640.790511006"
Sep 06 06:04:48.029236 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:04:48.029204    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-p9tsq"
Sep 06 06:05:13.384542 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:13.384510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:18.081198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:18.081169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wpf4d"
Sep 06 06:05:35.033196 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:35.033138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:35.098385 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:35.098344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:38.167379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.167341    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"]
Sep 06 06:05:38.170666 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.170643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.173450 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.173423    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 06:05:38.173548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.173448    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sq2v6\""
Sep 06 06:05:38.173548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.173516    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 06:05:38.179285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.179264    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"]
Sep 06 06:05:38.199328 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.199306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jjf4\" (UniqueName: \"kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.199421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.199351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.199421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.199375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.270556 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.270519    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"]
Sep 06 06:05:38.273573 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.273550    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.281846 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.281289    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"]
Sep 06 06:05:38.300445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.300635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.300635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.300635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.300635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhrt\" (UniqueName: \"kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.300635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jjf4\" (UniqueName: \"kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.300904 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.300904 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.300854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.308363 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.308335    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jjf4\" (UniqueName: \"kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.370232 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.370195    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"]
Sep 06 06:05:38.373645 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.373628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.381690 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.381664    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"]
Sep 06 06:05:38.401663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.401663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhrt\" (UniqueName: \"kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.401824 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.401824 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401751    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjhl4\" (UniqueName: \"kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.401990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401968    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.402041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.402005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.402041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.401976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.402288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.402272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.409348 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.409320    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhrt\" (UniqueName: \"kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.467807 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.467734    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"]
Sep 06 06:05:38.471119 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.471103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.479147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.479121    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"]
Sep 06 06:05:38.480094 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.480055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:38.502999 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.502976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.503117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjhl4\" (UniqueName: \"kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.503117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lknzv\" (UniqueName: \"kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.503460 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.503460 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.503622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.503622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.503786 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.503749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.511464 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.511443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjhl4\" (UniqueName: \"kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.584242 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.584205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:38.604612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.604580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lknzv\" (UniqueName: \"kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.604731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.604628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.604731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.604674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.605122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.605096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.605122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.605116    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.613071 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.613034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lknzv\" (UniqueName: \"kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.683588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.683548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:38.764827 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.764796    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"]
Sep 06 06:05:38.766984 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:05:38.766954    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8650a2fa_bdf9_4f64_9c3c_7963b6ceae5d.slice/crio-05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba WatchSource:0}: Error finding container 05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba: Status 404 returned error can't find the container with id 05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba
Sep 06 06:05:38.780952 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.780929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:38.858104 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.856954    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"]
Sep 06 06:05:38.863620 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:05:38.863595    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa3e3f3_f7d8_47c8_9d49_87f1087726ab.slice/crio-e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1 WatchSource:0}: Error finding container e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1: Status 404 returned error can't find the container with id e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1
Sep 06 06:05:38.956503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:38.956452    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"]
Sep 06 06:05:38.959109 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:05:38.959085    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378d2954_1778_4dff_ae41_728bd316b9cd.slice/crio-5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f WatchSource:0}: Error finding container 5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f: Status 404 returned error can't find the container with id 5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f
Sep 06 06:05:39.054263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.054239    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"]
Sep 06 06:05:39.114791 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:05:39.114759    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08498701_7fa3_4325_8a25_712d8ef5747e.slice/crio-145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0 WatchSource:0}: Error finding container 145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0: Status 404 returned error can't find the container with id 145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0
Sep 06 06:05:39.291616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.291525    2567 generic.go:356] "Generic (PLEG): container finished" podID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerID="097dfeccf88ab673e64a1e4ba1fc95eaf945b8423b6fd295c82f587d31870ec8" exitCode=0
Sep 06 06:05:39.291616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.291602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf" event={"ID":"faa3e3f3-f7d8-47c8-9d49-87f1087726ab","Type":"ContainerDied","Data":"097dfeccf88ab673e64a1e4ba1fc95eaf945b8423b6fd295c82f587d31870ec8"}
Sep 06 06:05:39.292171 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.291660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf" event={"ID":"faa3e3f3-f7d8-47c8-9d49-87f1087726ab","Type":"ContainerStarted","Data":"e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1"}
Sep 06 06:05:39.293191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.293140    2567 generic.go:356] "Generic (PLEG): container finished" podID="378d2954-1778-4dff-ae41-728bd316b9cd" containerID="42e214991a340e7073bacb1d23aff4fba3f46758f70edecfc4736969954bf03b" exitCode=0
Sep 06 06:05:39.293240 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.293211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km" event={"ID":"378d2954-1778-4dff-ae41-728bd316b9cd","Type":"ContainerDied","Data":"42e214991a340e7073bacb1d23aff4fba3f46758f70edecfc4736969954bf03b"}
Sep 06 06:05:39.293290 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.293255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km" event={"ID":"378d2954-1778-4dff-ae41-728bd316b9cd","Type":"ContainerStarted","Data":"5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f"}
Sep 06 06:05:39.294880 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.294850    2567 generic.go:356] "Generic (PLEG): container finished" podID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerID="fc03474d107c68dc74a041e7df63f6754812c992fb28114cdfe12c8ca396a827" exitCode=0
Sep 06 06:05:39.294989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.294886    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j" event={"ID":"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d","Type":"ContainerDied","Data":"fc03474d107c68dc74a041e7df63f6754812c992fb28114cdfe12c8ca396a827"}
Sep 06 06:05:39.295053 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.295033    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j" event={"ID":"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d","Type":"ContainerStarted","Data":"05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba"}
Sep 06 06:05:39.296825 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.296791    2567 generic.go:356] "Generic (PLEG): container finished" podID="08498701-7fa3-4325-8a25-712d8ef5747e" containerID="74146064f40272e20bc32a0954910b4b3f8386694547dc618d22ae81f6ae165a" exitCode=0
Sep 06 06:05:39.296902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.296844    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx" event={"ID":"08498701-7fa3-4325-8a25-712d8ef5747e","Type":"ContainerDied","Data":"74146064f40272e20bc32a0954910b4b3f8386694547dc618d22ae81f6ae165a"}
Sep 06 06:05:39.296902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:39.296892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx" event={"ID":"08498701-7fa3-4325-8a25-712d8ef5747e","Type":"ContainerStarted","Data":"145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0"}
Sep 06 06:05:41.308897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.308865    2567 generic.go:356] "Generic (PLEG): container finished" podID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerID="0a0aa6a26f40654aed15aea0baba16935360888fcfe49c11185a63d9943bbe34" exitCode=0
Sep 06 06:05:41.309346 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.308961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j" event={"ID":"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d","Type":"ContainerDied","Data":"0a0aa6a26f40654aed15aea0baba16935360888fcfe49c11185a63d9943bbe34"}
Sep 06 06:05:41.310924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.310899    2567 generic.go:356] "Generic (PLEG): container finished" podID="08498701-7fa3-4325-8a25-712d8ef5747e" containerID="932d65ff4e2b637456f6a4fa962249de9221786fdc2b2fee3e989d128fda5415" exitCode=0
Sep 06 06:05:41.311035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.310979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx" event={"ID":"08498701-7fa3-4325-8a25-712d8ef5747e","Type":"ContainerDied","Data":"932d65ff4e2b637456f6a4fa962249de9221786fdc2b2fee3e989d128fda5415"}
Sep 06 06:05:41.312971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.312952    2567 generic.go:356] "Generic (PLEG): container finished" podID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerID="2fb65c143ea54e4b61c774105cc21232a617dcc5fe4c68170dd4807dfa9a44a9" exitCode=0
Sep 06 06:05:41.313070 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.313028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf" event={"ID":"faa3e3f3-f7d8-47c8-9d49-87f1087726ab","Type":"ContainerDied","Data":"2fb65c143ea54e4b61c774105cc21232a617dcc5fe4c68170dd4807dfa9a44a9"}
Sep 06 06:05:41.315047 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.315024    2567 generic.go:356] "Generic (PLEG): container finished" podID="378d2954-1778-4dff-ae41-728bd316b9cd" containerID="e121382c0b366fda8ba4bdf41537ce2fb6c8b1e327c638b4e78c7e5a247f6540" exitCode=0
Sep 06 06:05:41.315129 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:41.315080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km" event={"ID":"378d2954-1778-4dff-ae41-728bd316b9cd","Type":"ContainerDied","Data":"e121382c0b366fda8ba4bdf41537ce2fb6c8b1e327c638b4e78c7e5a247f6540"}
Sep 06 06:05:42.321993 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.321960    2567 generic.go:356] "Generic (PLEG): container finished" podID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerID="50b26028d4e4139d4ebb75f011652c81814f59aadfb19666633aa2a6f8906d1b" exitCode=0
Sep 06 06:05:42.322434 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.322054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j" event={"ID":"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d","Type":"ContainerDied","Data":"50b26028d4e4139d4ebb75f011652c81814f59aadfb19666633aa2a6f8906d1b"}
Sep 06 06:05:42.323979 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.323956    2567 generic.go:356] "Generic (PLEG): container finished" podID="08498701-7fa3-4325-8a25-712d8ef5747e" containerID="70add5776296e7884118ea8085f303a98133fceeb8647fd2514cee0af77205f1" exitCode=0
Sep 06 06:05:42.324124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.324043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx" event={"ID":"08498701-7fa3-4325-8a25-712d8ef5747e","Type":"ContainerDied","Data":"70add5776296e7884118ea8085f303a98133fceeb8647fd2514cee0af77205f1"}
Sep 06 06:05:42.326158 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.326135    2567 generic.go:356] "Generic (PLEG): container finished" podID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerID="1656e98091d8f43758d14601c7ba9aede85b17df20fc56bed3f32f4d40693fd3" exitCode=0
Sep 06 06:05:42.326242 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.326219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf" event={"ID":"faa3e3f3-f7d8-47c8-9d49-87f1087726ab","Type":"ContainerDied","Data":"1656e98091d8f43758d14601c7ba9aede85b17df20fc56bed3f32f4d40693fd3"}
Sep 06 06:05:42.328402 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.328382    2567 generic.go:356] "Generic (PLEG): container finished" podID="378d2954-1778-4dff-ae41-728bd316b9cd" containerID="33fdf175c130f460e070755ee4f916909c81fd5cc3763005a54cd45bd6dcfc49" exitCode=0
Sep 06 06:05:42.328480 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:42.328424    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km" event={"ID":"378d2954-1778-4dff-ae41-728bd316b9cd","Type":"ContainerDied","Data":"33fdf175c130f460e070755ee4f916909c81fd5cc3763005a54cd45bd6dcfc49"}
Sep 06 06:05:43.484271 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.484235    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:43.503701 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.503671    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:43.523945 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.523921    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:43.528662 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.528640    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:43.544844 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.544815    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjhl4\" (UniqueName: \"kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4\") pod \"378d2954-1778-4dff-ae41-728bd316b9cd\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") "
Sep 06 06:05:43.544952 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.544856    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzhrt\" (UniqueName: \"kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt\") pod \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") "
Sep 06 06:05:43.544952 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.544892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle\") pod \"378d2954-1778-4dff-ae41-728bd316b9cd\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") "
Sep 06 06:05:43.545092 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.544959    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util\") pod \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") "
Sep 06 06:05:43.545092 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.545033    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle\") pod \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\" (UID: \"faa3e3f3-f7d8-47c8-9d49-87f1087726ab\") "
Sep 06 06:05:43.545092 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.545081    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util\") pod \"378d2954-1778-4dff-ae41-728bd316b9cd\" (UID: \"378d2954-1778-4dff-ae41-728bd316b9cd\") "
Sep 06 06:05:43.546106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.545897    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle" (OuterVolumeSpecName: "bundle") pod "faa3e3f3-f7d8-47c8-9d49-87f1087726ab" (UID: "faa3e3f3-f7d8-47c8-9d49-87f1087726ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.546759 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.546735    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle" (OuterVolumeSpecName: "bundle") pod "378d2954-1778-4dff-ae41-728bd316b9cd" (UID: "378d2954-1778-4dff-ae41-728bd316b9cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.547521 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.547479    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4" (OuterVolumeSpecName: "kube-api-access-vjhl4") pod "378d2954-1778-4dff-ae41-728bd316b9cd" (UID: "378d2954-1778-4dff-ae41-728bd316b9cd"). InnerVolumeSpecName "kube-api-access-vjhl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:05:43.547624 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.547566    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt" (OuterVolumeSpecName: "kube-api-access-dzhrt") pod "faa3e3f3-f7d8-47c8-9d49-87f1087726ab" (UID: "faa3e3f3-f7d8-47c8-9d49-87f1087726ab"). InnerVolumeSpecName "kube-api-access-dzhrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:05:43.550588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.550559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util" (OuterVolumeSpecName: "util") pod "faa3e3f3-f7d8-47c8-9d49-87f1087726ab" (UID: "faa3e3f3-f7d8-47c8-9d49-87f1087726ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.551757 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.551739    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util" (OuterVolumeSpecName: "util") pod "378d2954-1778-4dff-ae41-728bd316b9cd" (UID: "378d2954-1778-4dff-ae41-728bd316b9cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.646317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646233    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jjf4\" (UniqueName: \"kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4\") pod \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") "
Sep 06 06:05:43.646317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646289    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle\") pod \"08498701-7fa3-4325-8a25-712d8ef5747e\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") "
Sep 06 06:05:43.646529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646325    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lknzv\" (UniqueName: \"kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv\") pod \"08498701-7fa3-4325-8a25-712d8ef5747e\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") "
Sep 06 06:05:43.646529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util\") pod \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") "
Sep 06 06:05:43.646529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646377    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle\") pod \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\" (UID: \"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d\") "
Sep 06 06:05:43.646529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646398    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util\") pod \"08498701-7fa3-4325-8a25-712d8ef5747e\" (UID: \"08498701-7fa3-4325-8a25-712d8ef5747e\") "
Sep 06 06:05:43.646529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646530    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646545    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646557    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646568    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/378d2954-1778-4dff-ae41-728bd316b9cd-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646580    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjhl4\" (UniqueName: \"kubernetes.io/projected/378d2954-1778-4dff-ae41-728bd316b9cd-kube-api-access-vjhl4\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646596    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzhrt\" (UniqueName: \"kubernetes.io/projected/faa3e3f3-f7d8-47c8-9d49-87f1087726ab-kube-api-access-dzhrt\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.646921 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.646878    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle" (OuterVolumeSpecName: "bundle") pod "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" (UID: "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.647156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.647136    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle" (OuterVolumeSpecName: "bundle") pod "08498701-7fa3-4325-8a25-712d8ef5747e" (UID: "08498701-7fa3-4325-8a25-712d8ef5747e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.648843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.648822    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv" (OuterVolumeSpecName: "kube-api-access-lknzv") pod "08498701-7fa3-4325-8a25-712d8ef5747e" (UID: "08498701-7fa3-4325-8a25-712d8ef5747e"). InnerVolumeSpecName "kube-api-access-lknzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:05:43.648938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.648859    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4" (OuterVolumeSpecName: "kube-api-access-6jjf4") pod "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" (UID: "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d"). InnerVolumeSpecName "kube-api-access-6jjf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:05:43.652422 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.652402    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util" (OuterVolumeSpecName: "util") pod "08498701-7fa3-4325-8a25-712d8ef5747e" (UID: "08498701-7fa3-4325-8a25-712d8ef5747e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.652937 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.652918    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util" (OuterVolumeSpecName: "util") pod "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" (UID: "8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:05:43.747948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747915    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.747948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747943    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jjf4\" (UniqueName: \"kubernetes.io/projected/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-kube-api-access-6jjf4\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.747948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747953    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08498701-7fa3-4325-8a25-712d8ef5747e-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.748331 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747962    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lknzv\" (UniqueName: \"kubernetes.io/projected/08498701-7fa3-4325-8a25-712d8ef5747e-kube-api-access-lknzv\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.748331 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747974    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-util\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:43.748331 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:43.747982    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d-bundle\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:05:44.339674 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.339639    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf" event={"ID":"faa3e3f3-f7d8-47c8-9d49-87f1087726ab","Type":"ContainerDied","Data":"e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1"}
Sep 06 06:05:44.339942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.339692    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e834bcc63002d3809625f4d2f448d433b0b5ba50517b882d2a94fe9787f4f9c1"
Sep 06 06:05:44.339942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.339645    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs78jf"
Sep 06 06:05:44.341641 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.341609    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km" event={"ID":"378d2954-1778-4dff-ae41-728bd316b9cd","Type":"ContainerDied","Data":"5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f"}
Sep 06 06:05:44.341641 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.341627    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcq4km"
Sep 06 06:05:44.341821 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.341657    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a231386cd332252242f956b03a7afe4a77c0a15d5b1792e481b6b1bc099074f"
Sep 06 06:05:44.343515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.343498    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j"
Sep 06 06:05:44.343635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.343532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0n6b6j" event={"ID":"8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d","Type":"ContainerDied","Data":"05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba"}
Sep 06 06:05:44.343635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.343574    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05c35cff2cbf774e780ad6f7d5d93809e84c1b86e9257b500ad0fe5a9a719bba"
Sep 06 06:05:44.345511 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.345483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx" event={"ID":"08498701-7fa3-4325-8a25-712d8ef5747e","Type":"ContainerDied","Data":"145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0"}
Sep 06 06:05:44.345606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.345518    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="145c686a5f17a72b33edeb2043f860062f30010d7633d97a2c92406a62adc9e0"
Sep 06 06:05:44.345606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:44.345492    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053scwrx"
Sep 06 06:05:47.573528 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:47.573488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:50.171137 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:50.171104    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:50.571337 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:50.571302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:50.976281 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:50.976190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:53.583258 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583216    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"]
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583594    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="extract"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583610    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="extract"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583622    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583630    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583642    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="pull"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583650    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="pull"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583659    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="extract"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583666    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="extract"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583682    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583689    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583699    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583707    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="util"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583715    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="pull"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583724    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="pull"
Sep 06 06:05:53.583729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583734    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583744    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583755    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="pull"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583762    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="pull"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583773    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583781    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583799    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="util"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583807    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="util"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583820    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="pull"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583828    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="pull"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583900    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="faa3e3f3-f7d8-47c8-9d49-87f1087726ab" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583911    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="378d2954-1778-4dff-ae41-728bd316b9cd" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583923    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8650a2fa-bdf9-4f64-9c3c-7963b6ceae5d" containerName="extract"
Sep 06 06:05:53.584453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.583932    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="08498701-7fa3-4325-8a25-712d8ef5747e" containerName="extract"
Sep 06 06:05:53.588563 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.588539    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.592423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.592403    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 06:05:53.592549 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.592527    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 06:05:53.592622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.592537    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m6mpq\""
Sep 06 06:05:53.605154 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.605118    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"]
Sep 06 06:05:53.737945 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.737908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jmn8\" (UniqueName: \"kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.737945 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.737946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.838973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.838865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.838973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.838980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jmn8\" (UniqueName: \"kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.839325 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.839303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.856252 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.856221    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jmn8\" (UniqueName: \"kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8\") pod \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:53.899011 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:53.898969    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:54.142336 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:54.142247    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:54.188611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:54.188579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"]
Sep 06 06:05:54.192354 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:05:54.192324    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749ebbab_0417_4e03_b992_2873b5d12e59.slice/crio-7eeffc5d0bde51bfeb3f8f83e166e5a37e45161e99353a9b06dbc9ddd722ceab WatchSource:0}: Error finding container 7eeffc5d0bde51bfeb3f8f83e166e5a37e45161e99353a9b06dbc9ddd722ceab: Status 404 returned error can't find the container with id 7eeffc5d0bde51bfeb3f8f83e166e5a37e45161e99353a9b06dbc9ddd722ceab
Sep 06 06:05:54.243810 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:54.243763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:54.391839 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:54.391805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf" event={"ID":"749ebbab-0417-4e03-b992-2873b5d12e59","Type":"ContainerStarted","Data":"7eeffc5d0bde51bfeb3f8f83e166e5a37e45161e99353a9b06dbc9ddd722ceab"}
Sep 06 06:05:55.036937 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:55.036896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:55.137755 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:55.137717    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:55.965139 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:55.965097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:56.379904 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:56.379844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:56.616347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:56.616315    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:56.721106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:56.720992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:58.009054 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:58.009018    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:05:58.421656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:58.421624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf" event={"ID":"749ebbab-0417-4e03-b992-2873b5d12e59","Type":"ContainerStarted","Data":"8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423"}
Sep 06 06:05:58.421843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:58.421700    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:05:58.442260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:58.442204    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf" podStartSLOduration=1.431231975 podStartE2EDuration="5.442186486s" totalImagesPullingTime="4.010954511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:05:58.440788466 +0000 UTC m=+712.144341201" watchObservedRunningTime="2026-09-06 06:05:58.442186486 +0000 UTC m=+712.145739221"
Sep 06 06:05:59.982353 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:05:59.982277    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:00.083474 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:00.083433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:00.765387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:00.765342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:00.881897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:00.881857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:03.769694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:03.769657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:03.870202 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:03.870164    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:04.173539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:04.173504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:04.273851 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:04.273810    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:04.675923 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:04.675881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:04.780195 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:04.780161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:09.430181 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:09.430151    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:10.102273 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:10.102243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:10.202872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:10.202835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:11.479159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.479123    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"]
Sep 06 06:06:11.479649 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.479400    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" containerName="manager" containerID="cri-o://8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423" gracePeriod=2
Sep 06 06:06:11.489254 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.489224    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"]
Sep 06 06:06:11.494120 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:11.494087    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"749ebbab-0417-4e03-b992-2873b5d12e59\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-vlhwf\": pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:11.500612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.500588    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:11.500851 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.500838    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" containerName="manager"
Sep 06 06:06:11.500851 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.500851    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" containerName="manager"
Sep 06 06:06:11.500968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.500915    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" containerName="manager"
Sep 06 06:06:11.540316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.540278    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:11.540428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.540398    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.542801 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:11.542773    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:11.681551 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.681521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j474\" (UniqueName: \"kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.681697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.681625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.744340 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.744280    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:11.746689 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:11.746662    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:11.782128 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.782096    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.782228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.782138    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j474\" (UniqueName: \"kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.782450 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.782432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.796699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.796678    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j474\" (UniqueName: \"kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpq2j\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.883282 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.883252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jmn8\" (UniqueName: \"kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8\") pod \"749ebbab-0417-4e03-b992-2873b5d12e59\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") "
Sep 06 06:06:11.883444 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.883324    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume\") pod \"749ebbab-0417-4e03-b992-2873b5d12e59\" (UID: \"749ebbab-0417-4e03-b992-2873b5d12e59\") "
Sep 06 06:06:11.883715 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.883693    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "749ebbab-0417-4e03-b992-2873b5d12e59" (UID: "749ebbab-0417-4e03-b992-2873b5d12e59"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:06:11.885490 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.885471    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8" (OuterVolumeSpecName: "kube-api-access-7jmn8") pod "749ebbab-0417-4e03-b992-2873b5d12e59" (UID: "749ebbab-0417-4e03-b992-2873b5d12e59"). InnerVolumeSpecName "kube-api-access-7jmn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:06:11.918056 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.918025    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:11.984324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.984193    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jmn8\" (UniqueName: \"kubernetes.io/projected/749ebbab-0417-4e03-b992-2873b5d12e59-kube-api-access-7jmn8\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:11.984324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:11.984224    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/749ebbab-0417-4e03-b992-2873b5d12e59-extensions-socket-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:12.213846 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.213818    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:12.216337 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:06:12.216306    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf8ec4d_21eb_4d2a_abb3_23cf78fa9568.slice/crio-3d5cdbe3f7a05f2b339a507a5f4128e93b8013222502c29a839171419325120e WatchSource:0}: Error finding container 3d5cdbe3f7a05f2b339a507a5f4128e93b8013222502c29a839171419325120e: Status 404 returned error can't find the container with id 3d5cdbe3f7a05f2b339a507a5f4128e93b8013222502c29a839171419325120e
Sep 06 06:06:12.493687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.493612    2567 generic.go:356] "Generic (PLEG): container finished" podID="749ebbab-0417-4e03-b992-2873b5d12e59" containerID="8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423" exitCode=0
Sep 06 06:06:12.493687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.493654    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:12.494176 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.493717    2567 scope.go:117] "RemoveContainer" containerID="8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423"
Sep 06 06:06:12.495738 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.495712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" event={"ID":"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568","Type":"ContainerStarted","Data":"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"}
Sep 06 06:06:12.495820 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.495770    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" event={"ID":"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568","Type":"ContainerStarted","Data":"3d5cdbe3f7a05f2b339a507a5f4128e93b8013222502c29a839171419325120e"}
Sep 06 06:06:12.495820 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.495803    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:12.496200 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:12.496176    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:12.503035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.503019    2567 scope.go:117] "RemoveContainer" containerID="8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423"
Sep 06 06:06:12.503296 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:12.503279    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423\": container with ID starting with 8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423 not found: ID does not exist" containerID="8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423"
Sep 06 06:06:12.503357 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.503301    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423"} err="failed to get container status \"8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423\": rpc error: code = NotFound desc = could not find container \"8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423\": container with ID starting with 8eaa5b2cb5f7e3fa1ee8ffdec2e3934cf951ae99a6edd856db00fc7d6e7a7423 not found: ID does not exist"
Sep 06 06:06:12.522351 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.522304    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" podStartSLOduration=1.522272428 podStartE2EDuration="1.522272428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:06:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:06:12.520159258 +0000 UTC m=+726.223711995" watchObservedRunningTime="2026-09-06 06:06:12.522272428 +0000 UTC m=+726.225825164"
Sep 06 06:06:12.522448 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:12.522379    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:12.524328 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:12.524305    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:12.615878 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.615845    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:06:12.647465 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.647428    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:06:12.647618 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.647561    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.650303 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:12.650276    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vlhwf\" is forbidden: User \"system:node:ip-10-0-129-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-33.ec2.internal' and this object" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vlhwf"
Sep 06 06:06:12.786858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.786791    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749ebbab-0417-4e03-b992-2873b5d12e59" path="/var/lib/kubelet/pods/749ebbab-0417-4e03-b992-2873b5d12e59/volumes"
Sep 06 06:06:12.790314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.790296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.790379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.790336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65lv\" (UniqueName: \"kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.891112 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.891080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65lv\" (UniqueName: \"kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.891288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.891180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.891599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.891579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.900124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.900104    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65lv\" (UniqueName: \"kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6lt86\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:12.958016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:12.957976    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:13.239734 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:13.239691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:06:13.245511 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:06:13.245484    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb06f7e1_01ab_4fdd_a69c_c9f8ead58970.slice/crio-527d47c4ad827fced4e4a59fa6f602e46f92015b9432305fdbc6e9ad4f5f5b7f WatchSource:0}: Error finding container 527d47c4ad827fced4e4a59fa6f602e46f92015b9432305fdbc6e9ad4f5f5b7f: Status 404 returned error can't find the container with id 527d47c4ad827fced4e4a59fa6f602e46f92015b9432305fdbc6e9ad4f5f5b7f
Sep 06 06:06:13.502998 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:13.502901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" event={"ID":"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970","Type":"ContainerStarted","Data":"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"}
Sep 06 06:06:13.502998 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:13.502965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" event={"ID":"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970","Type":"ContainerStarted","Data":"527d47c4ad827fced4e4a59fa6f602e46f92015b9432305fdbc6e9ad4f5f5b7f"}
Sep 06 06:06:13.503408 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:13.503127    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:13.534888 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:13.534808    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" podStartSLOduration=1.5347923890000001 podStartE2EDuration="1.534792389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:06:13.53341757 +0000 UTC m=+727.236970305" watchObservedRunningTime="2026-09-06 06:06:13.534792389 +0000 UTC m=+727.238345125"
Sep 06 06:06:16.584905 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:16.584875    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:16.682439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:16.682403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:17.844655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:17.844621    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:17.943507 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:17.943456    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:19.179867 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:19.179828    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:19.280835 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:19.280795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:22.999525 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:22.999488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:23.096687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:23.096646    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:23.197491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:23.197464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:23.299125 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:23.299016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:23.505563 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:23.505534    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:24.194310 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.194274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:24.251492 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.251461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:24.510666 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.510586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:06:24.587501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.587472    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:24.587703 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.587684    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" podUID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" containerName="manager" containerID="cri-o://5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a" gracePeriod=10
Sep 06 06:06:24.846115 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.846093    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:24.877820 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.877788    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j474\" (UniqueName: \"kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474\") pod \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") "
Sep 06 06:06:24.877989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.877849    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume\") pod \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\" (UID: \"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568\") "
Sep 06 06:06:24.878368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.878336    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" (UID: "9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:06:24.879877 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.879854    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474" (OuterVolumeSpecName: "kube-api-access-8j474") pod "9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" (UID: "9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568"). InnerVolumeSpecName "kube-api-access-8j474". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:06:24.979082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.979002    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j474\" (UniqueName: \"kubernetes.io/projected/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-kube-api-access-8j474\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:24.979082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:24.979031    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568-extensions-socket-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:25.561675 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.561644    2567 generic.go:356] "Generic (PLEG): container finished" podID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" containerID="5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a" exitCode=0
Sep 06 06:06:25.562098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.561683    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"
Sep 06 06:06:25.562098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.561730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" event={"ID":"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568","Type":"ContainerDied","Data":"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"}
Sep 06 06:06:25.562098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.561783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j" event={"ID":"9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568","Type":"ContainerDied","Data":"3d5cdbe3f7a05f2b339a507a5f4128e93b8013222502c29a839171419325120e"}
Sep 06 06:06:25.562098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.561806    2567 scope.go:117] "RemoveContainer" containerID="5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"
Sep 06 06:06:25.570408 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.570392    2567 scope.go:117] "RemoveContainer" containerID="5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"
Sep 06 06:06:25.570664 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:25.570646    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a\": container with ID starting with 5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a not found: ID does not exist" containerID="5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"
Sep 06 06:06:25.570709 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.570672    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a"} err="failed to get container status \"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a\": rpc error: code = NotFound desc = could not find container \"5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a\": container with ID starting with 5c4ea06b83040daeac5a5630ca30da32a853a5ebf872e41e8ac29bea56059a6a not found: ID does not exist"
Sep 06 06:06:25.586485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.586462    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:25.589997 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:25.589975    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpq2j"]
Sep 06 06:06:26.785705 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:26.785671    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" path="/var/lib/kubelet/pods/9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568/volumes"
Sep 06 06:06:31.036434 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:31.036399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:31.087423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:31.087388    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:37.594609 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:37.594573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:37.693836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:37.693798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:40.934608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:40.934578    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:41.118522 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:41.118487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:43.245202 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.245166    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:43.245578 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.245449    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" containerName="manager"
Sep 06 06:06:43.245578 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.245462    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" containerName="manager"
Sep 06 06:06:43.245578 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.245524    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf8ec4d-21eb-4d2a-abb3-23cf78fa9568" containerName="manager"
Sep 06 06:06:43.246862 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.246826    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:43.248396 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.248380    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.250677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.250660    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 06 06:06:43.250873 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.250860    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9w9zr\""
Sep 06 06:06:43.256454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.256433    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:43.319627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.319589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.319802 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.319633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrcgh\" (UniqueName: \"kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.341413 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.341379    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:43.343242 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.343204    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:43.396625 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.396590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:43.421130 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.421103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.421258 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.421135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrcgh\" (UniqueName: \"kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.421701 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.421682    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.428597 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.428577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrcgh\" (UniqueName: \"kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh\") pod \"limitador-limitador-859c4d96d6-6xtkt\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.558778 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.558747    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:43.832122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:43.832091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:43.834605 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:06:43.834571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf41db8f_617b_4efc_80ef_c356f009d372.slice/crio-bc092441a1f4efc21e5c0ea9800e69f11054961bb4448c4fd8dab482c1460a16 WatchSource:0}: Error finding container bc092441a1f4efc21e5c0ea9800e69f11054961bb4448c4fd8dab482c1460a16: Status 404 returned error can't find the container with id bc092441a1f4efc21e5c0ea9800e69f11054961bb4448c4fd8dab482c1460a16
Sep 06 06:06:44.083164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:44.083078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:44.170813 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:44.170772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:44.659748 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:44.659709    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" event={"ID":"bf41db8f-617b-4efc-80ef-c356f009d372","Type":"ContainerStarted","Data":"bc092441a1f4efc21e5c0ea9800e69f11054961bb4448c4fd8dab482c1460a16"}
Sep 06 06:06:45.666470 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:45.666435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" event={"ID":"bf41db8f-617b-4efc-80ef-c356f009d372","Type":"ContainerStarted","Data":"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"}
Sep 06 06:06:45.666806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:45.666573    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:45.681363 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:45.681293    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" podStartSLOduration=1.4081344869999999 podStartE2EDuration="2.681275675s" totalImagesPullingTime="1.273141188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:06:45.681072585 +0000 UTC m=+759.384625314" watchObservedRunningTime="2026-09-06 06:06:45.681275675 +0000 UTC m=+759.384828411"
Sep 06 06:06:56.331341 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.331295    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:56.331834 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.331502    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" podUID="bf41db8f-617b-4efc-80ef-c356f009d372" containerName="limitador" containerID="cri-o://f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9" gracePeriod=30
Sep 06 06:06:56.332277 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.332255    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:56.867794 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.867770    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:56.932335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.932254    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrcgh\" (UniqueName: \"kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh\") pod \"bf41db8f-617b-4efc-80ef-c356f009d372\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") "
Sep 06 06:06:56.932335 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.932333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file\") pod \"bf41db8f-617b-4efc-80ef-c356f009d372\" (UID: \"bf41db8f-617b-4efc-80ef-c356f009d372\") "
Sep 06 06:06:56.932655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.932633    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file" (OuterVolumeSpecName: "config-file") pod "bf41db8f-617b-4efc-80ef-c356f009d372" (UID: "bf41db8f-617b-4efc-80ef-c356f009d372"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:06:56.934358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:56.934332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh" (OuterVolumeSpecName: "kube-api-access-xrcgh") pod "bf41db8f-617b-4efc-80ef-c356f009d372" (UID: "bf41db8f-617b-4efc-80ef-c356f009d372"). InnerVolumeSpecName "kube-api-access-xrcgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:06:57.032956 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.032914    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrcgh\" (UniqueName: \"kubernetes.io/projected/bf41db8f-617b-4efc-80ef-c356f009d372-kube-api-access-xrcgh\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:57.032956 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.032949    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf41db8f-617b-4efc-80ef-c356f009d372-config-file\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:06:57.723000 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.722966    2567 generic.go:356] "Generic (PLEG): container finished" podID="bf41db8f-617b-4efc-80ef-c356f009d372" containerID="f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9" exitCode=0
Sep 06 06:06:57.723439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.723013    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"
Sep 06 06:06:57.723439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.723041    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" event={"ID":"bf41db8f-617b-4efc-80ef-c356f009d372","Type":"ContainerDied","Data":"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"}
Sep 06 06:06:57.723439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.723105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6xtkt" event={"ID":"bf41db8f-617b-4efc-80ef-c356f009d372","Type":"ContainerDied","Data":"bc092441a1f4efc21e5c0ea9800e69f11054961bb4448c4fd8dab482c1460a16"}
Sep 06 06:06:57.723439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.723137    2567 scope.go:117] "RemoveContainer" containerID="f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"
Sep 06 06:06:57.732470 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.732449    2567 scope.go:117] "RemoveContainer" containerID="f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"
Sep 06 06:06:57.732749 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:06:57.732730    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9\": container with ID starting with f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9 not found: ID does not exist" containerID="f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"
Sep 06 06:06:57.732802 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.732757    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9"} err="failed to get container status \"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9\": rpc error: code = NotFound desc = could not find container \"f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9\": container with ID starting with f650d1e6a9de6c26d0314259a964466a499087098b05f1707ba49d85c98b8bb9 not found: ID does not exist"
Sep 06 06:06:57.743748 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.743719    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:57.747639 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:57.747617    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6xtkt"]
Sep 06 06:06:58.785188 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:58.785152    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf41db8f-617b-4efc-80ef-c356f009d372" path="/var/lib/kubelet/pods/bf41db8f-617b-4efc-80ef-c356f009d372/volumes"
Sep 06 06:06:59.319835 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.319801    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:59.420137 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.420101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:59.741142 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.741049    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"]
Sep 06 06:06:59.741374 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.741360    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf41db8f-617b-4efc-80ef-c356f009d372" containerName="limitador"
Sep 06 06:06:59.741418 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.741376    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf41db8f-617b-4efc-80ef-c356f009d372" containerName="limitador"
Sep 06 06:06:59.741458 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.741430    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf41db8f-617b-4efc-80ef-c356f009d372" containerName="limitador"
Sep 06 06:06:59.745893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.745877    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.748626 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.748599    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 06:06:59.748727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.748694    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 06:06:59.749968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.749947    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 06 06:06:59.750271 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.750249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lrf79\""
Sep 06 06:06:59.752920 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.752899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"]
Sep 06 06:06:59.773296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.773254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:06:59.853622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.853586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g95mv\" (UniqueName: \"kubernetes.io/projected/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-kube-api-access-g95mv\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.853972 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.853702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-data\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.954925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.954884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-data\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.954925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.954932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95mv\" (UniqueName: \"kubernetes.io/projected/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-kube-api-access-g95mv\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.955346 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.955322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-data\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:06:59.964910 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:06:59.964880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95mv\" (UniqueName: \"kubernetes.io/projected/d0fb8daa-3e1b-4117-9121-7430ea2c9f9e-kube-api-access-g95mv\") pod \"postgres-69fb4594cf-59tkk\" (UID: \"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:07:00.058154 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:00.058125    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:07:00.326314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:00.326288    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"]
Sep 06 06:07:00.327963 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:07:00.327921    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fb8daa_3e1b_4117_9121_7430ea2c9f9e.slice/crio-19787aa807cef16d7d01caeb3cc1e2370008098a557437844dc738abed608854 WatchSource:0}: Error finding container 19787aa807cef16d7d01caeb3cc1e2370008098a557437844dc738abed608854: Status 404 returned error can't find the container with id 19787aa807cef16d7d01caeb3cc1e2370008098a557437844dc738abed608854
Sep 06 06:07:00.740468 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:00.740383    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk" event={"ID":"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e","Type":"ContainerStarted","Data":"19787aa807cef16d7d01caeb3cc1e2370008098a557437844dc738abed608854"}
Sep 06 06:07:05.773279 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:05.773240    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk" event={"ID":"d0fb8daa-3e1b-4117-9121-7430ea2c9f9e","Type":"ContainerStarted","Data":"c5c3e501f4e05f6e4a860834e6f94463db5c5e5d2c217725df3d277b470a9583"}
Sep 06 06:07:05.773647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:05.773311    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:07:05.790305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:05.790257    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk" podStartSLOduration=1.897486489 podStartE2EDuration="6.790242047s" totalImagesPullingTime="4.892755558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:07:05.788175254 +0000 UTC m=+779.491728006" watchObservedRunningTime="2026-09-06 06:07:05.790242047 +0000 UTC m=+779.493794783"
Sep 06 06:07:11.806164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:11.806132    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-59tkk"
Sep 06 06:07:12.108582 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:12.108497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:12.387120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:12.387007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.621621 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.621579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.641259 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.641229    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:19.658454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.658421    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:19.658595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.658548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.661347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.661322    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 06:07:19.661481 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.661323    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xhzg2\""
Sep 06 06:07:19.661481 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.661374    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 06:07:19.661591 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.661498    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 06:07:19.718338 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.718301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.808715 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.808683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.812963 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.812938    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:07:19.814972 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.814953    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:19.824603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.824578    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:07:19.827670 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.827644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rj68\" (UniqueName: \"kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.827800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.827769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.827863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.827828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.866026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.865972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.921318 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.921218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:19.928934 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.928907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:19.929051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.928942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr7rd\" (UniqueName: \"kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:19.929051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.928997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:19.929051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.929028    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.929167 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.929056    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.929167 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.929107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rj68\" (UniqueName: \"kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.931561 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.931541    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.931561 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.931552    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.937292 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.937266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rj68\" (UniqueName: \"kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68\") pod \"maas-controller-6cfb64dc6c-gqffc\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") " pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:19.970025 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:19.969984    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:20.029780 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.029727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.029926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.029786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr7rd\" (UniqueName: \"kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.029926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.029836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.032882 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.032851    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.033018 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.032960    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.038447 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.038422    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr7rd\" (UniqueName: \"kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd\") pod \"maas-controller-55947c9784-n24nx\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") " pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.126136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.126104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:20.238671 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.238646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:20.239754 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:07:20.239724    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2e7731_ed9b_4e15_aab2_f2da501ce886.slice/crio-3c753f644766af68debe99f2b97638f5ad9b55d7c6ab2d872e3d000606ee1a38 WatchSource:0}: Error finding container 3c753f644766af68debe99f2b97638f5ad9b55d7c6ab2d872e3d000606ee1a38: Status 404 returned error can't find the container with id 3c753f644766af68debe99f2b97638f5ad9b55d7c6ab2d872e3d000606ee1a38
Sep 06 06:07:20.404549 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.404521    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:07:20.844406 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.844368    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" event={"ID":"cf2e7731-ed9b-4e15-aab2-f2da501ce886","Type":"ContainerStarted","Data":"3c753f644766af68debe99f2b97638f5ad9b55d7c6ab2d872e3d000606ee1a38"}
Sep 06 06:07:20.845757 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:20.845733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-n24nx" event={"ID":"3bc95d52-1024-48e8-934b-fa4c4d1734d7","Type":"ContainerStarted","Data":"f5b2faecfe0209240f1d05a34df2cb73d3d49ccfd83aa94f963f6427ee869151"}
Sep 06 06:07:23.017095 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.017037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:23.117970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.117894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:23.863154 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.863101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-n24nx" event={"ID":"3bc95d52-1024-48e8-934b-fa4c4d1734d7","Type":"ContainerStarted","Data":"983cd9a58ab1bcf8da4e36509e40f531e8c574fe7030bcfb78fc2399fff5abd1"}
Sep 06 06:07:23.863336 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.863227    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:23.864743 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.864721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" event={"ID":"cf2e7731-ed9b-4e15-aab2-f2da501ce886","Type":"ContainerStarted","Data":"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"}
Sep 06 06:07:23.864869 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.864842    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:23.882150 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.882099    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55947c9784-n24nx" podStartSLOduration=2.4287912560000002 podStartE2EDuration="4.882084884s" totalImagesPullingTime="2.453293628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:07:23.880185181 +0000 UTC m=+797.583737945" watchObservedRunningTime="2026-09-06 06:07:23.882084884 +0000 UTC m=+797.585637632"
Sep 06 06:07:23.896831 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:23.896783    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" podStartSLOduration=2.288758624 podStartE2EDuration="4.896768583s" totalImagesPullingTime="2.608009959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:07:23.89588246 +0000 UTC m=+797.599435189" watchObservedRunningTime="2026-09-06 06:07:23.896768583 +0000 UTC m=+797.600321318"
Sep 06 06:07:34.871697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:34.871624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:34.871697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:34.871693    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:07:34.931971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:34.931940    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:34.932159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:34.932133    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" podUID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" containerName="manager" containerID="cri-o://643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5" gracePeriod=30
Sep 06 06:07:35.170228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.170198    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:35.246973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.246939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert\") pod \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") "
Sep 06 06:07:35.246973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.246976    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs\") pod \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") "
Sep 06 06:07:35.247224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.247045    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rj68\" (UniqueName: \"kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68\") pod \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\" (UID: \"cf2e7731-ed9b-4e15-aab2-f2da501ce886\") "
Sep 06 06:07:35.249224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.249193    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf2e7731-ed9b-4e15-aab2-f2da501ce886" (UID: "cf2e7731-ed9b-4e15-aab2-f2da501ce886"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:07:35.249347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.249231    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cf2e7731-ed9b-4e15-aab2-f2da501ce886" (UID: "cf2e7731-ed9b-4e15-aab2-f2da501ce886"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:07:35.249347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.249294    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68" (OuterVolumeSpecName: "kube-api-access-7rj68") pod "cf2e7731-ed9b-4e15-aab2-f2da501ce886" (UID: "cf2e7731-ed9b-4e15-aab2-f2da501ce886"). InnerVolumeSpecName "kube-api-access-7rj68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:07:35.348393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.348351    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-webhook-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:07:35.348393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.348383    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf2e7731-ed9b-4e15-aab2-f2da501ce886-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:07:35.348393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.348393    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rj68\" (UniqueName: \"kubernetes.io/projected/cf2e7731-ed9b-4e15-aab2-f2da501ce886-kube-api-access-7rj68\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:07:35.923230 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.923195    2567 generic.go:356] "Generic (PLEG): container finished" podID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" containerID="643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5" exitCode=0
Sep 06 06:07:35.923612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.923245    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc"
Sep 06 06:07:35.923612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.923275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" event={"ID":"cf2e7731-ed9b-4e15-aab2-f2da501ce886","Type":"ContainerDied","Data":"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"}
Sep 06 06:07:35.923612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.923327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfb64dc6c-gqffc" event={"ID":"cf2e7731-ed9b-4e15-aab2-f2da501ce886","Type":"ContainerDied","Data":"3c753f644766af68debe99f2b97638f5ad9b55d7c6ab2d872e3d000606ee1a38"}
Sep 06 06:07:35.923612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.923349    2567 scope.go:117] "RemoveContainer" containerID="643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"
Sep 06 06:07:35.932417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.932401    2567 scope.go:117] "RemoveContainer" containerID="643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"
Sep 06 06:07:35.932669 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:07:35.932653    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5\": container with ID starting with 643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5 not found: ID does not exist" containerID="643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"
Sep 06 06:07:35.932713 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.932677    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5"} err="failed to get container status \"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5\": rpc error: code = NotFound desc = could not find container \"643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5\": container with ID starting with 643bf35bf73e6d7228d6c49d816b639cc4997475d2cf47b7aaacb51e4980d0c5 not found: ID does not exist"
Sep 06 06:07:35.945169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.945142    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:35.948768 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:35.948747    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cfb64dc6c-gqffc"]
Sep 06 06:07:36.788878 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:36.788849    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" path="/var/lib/kubelet/pods/cf2e7731-ed9b-4e15-aab2-f2da501ce886/volumes"
Sep 06 06:07:54.613119 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:54.613080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:54.714470 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:54.714431    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.014644 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.014608    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.117781 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.117745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.172106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.172053    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:07:56.172394 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.172383    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" containerName="manager"
Sep 06 06:07:56.172440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.172397    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" containerName="manager"
Sep 06 06:07:56.172474 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.172461    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf2e7731-ed9b-4e15-aab2-f2da501ce886" containerName="manager"
Sep 06 06:07:56.203374 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.203335    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:07:56.203529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.203435    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.207694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.207672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 06:07:56.207694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.207688    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vd8t\""
Sep 06 06:07:56.208199 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.207674    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 06 06:07:56.315866 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.315833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.316025 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.315890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm5gl\" (UniqueName: \"kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.316025 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.315959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.316025 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.315999    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.416806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.416773    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.416806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.416811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.417020 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.416892    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.417020 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:07:56.416928    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 06 06:07:56.417020 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.416956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm5gl\" (UniqueName: \"kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.417020 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:07:56.417000    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls podName:641aaab5-0cd4-4016-bd88-fb349a6bfed6 nodeName:}" failed. No retries permitted until 2026-09-06 06:07:56.9169773 +0000 UTC m=+830.620530021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls") pod "maas-api-76f875657b-fd6jd" (UID: "641aaab5-0cd4-4016-bd88-fb349a6bfed6") : secret "maas-api-serving-cert" not found
Sep 06 06:07:56.417546 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.417523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.419421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.419397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.427627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.427599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm5gl\" (UniqueName: \"kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.446529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.446494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.553781 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.553737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.645974 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.645883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:07:56.711451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.711414    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"]
Sep 06 06:07:56.714241 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.714224    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.716897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.716874    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 06 06:07:56.717194 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.717179    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r7msj\""
Sep 06 06:07:56.723461 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.723434    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"]
Sep 06 06:07:56.819821 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.819780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.819996 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.819875    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzv2x\" (UniqueName: \"kubernetes.io/projected/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-kube-api-access-hzv2x\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.921386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.921280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.921386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.921382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzv2x\" (UniqueName: \"kubernetes.io/projected/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-kube-api-access-hzv2x\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.921624 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.921440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.922281 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.922254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:56.923932 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.923909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") pod \"maas-api-76f875657b-fd6jd\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") " pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:56.929454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:56.929433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzv2x\" (UniqueName: \"kubernetes.io/projected/9c49b2e6-4c6a-4cb7-911e-ecbb553b8121-kube-api-access-hzv2x\") pod \"payload-pre-processing-59fbc47bf8-8rpm6\" (UID: \"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:57.027009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:57.026979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:07:57.113837 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:57.113797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:07:57.295141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:57.295114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"]
Sep 06 06:07:57.296418 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:07:57.296392    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c49b2e6_4c6a_4cb7_911e_ecbb553b8121.slice/crio-e615fedff2c028f3ae40f73a88e3e41fce3f274c235cd33132edb8695bdfcd63 WatchSource:0}: Error finding container e615fedff2c028f3ae40f73a88e3e41fce3f274c235cd33132edb8695bdfcd63: Status 404 returned error can't find the container with id e615fedff2c028f3ae40f73a88e3e41fce3f274c235cd33132edb8695bdfcd63
Sep 06 06:07:57.387251 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:57.387207    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:07:57.389745 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:07:57.389719    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641aaab5_0cd4_4016_bd88_fb349a6bfed6.slice/crio-0c74338e7340f50e5c634637e8da6d4d4a8e8d4bc876afe701bdc4b2555dacb4 WatchSource:0}: Error finding container 0c74338e7340f50e5c634637e8da6d4d4a8e8d4bc876afe701bdc4b2555dacb4: Status 404 returned error can't find the container with id 0c74338e7340f50e5c634637e8da6d4d4a8e8d4bc876afe701bdc4b2555dacb4
Sep 06 06:07:58.037704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:58.037653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" event={"ID":"641aaab5-0cd4-4016-bd88-fb349a6bfed6","Type":"ContainerStarted","Data":"0c74338e7340f50e5c634637e8da6d4d4a8e8d4bc876afe701bdc4b2555dacb4"}
Sep 06 06:07:58.039466 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:07:58.039400    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6" event={"ID":"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121","Type":"ContainerStarted","Data":"e615fedff2c028f3ae40f73a88e3e41fce3f274c235cd33132edb8695bdfcd63"}
Sep 06 06:08:00.050596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.050558    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" event={"ID":"641aaab5-0cd4-4016-bd88-fb349a6bfed6","Type":"ContainerStarted","Data":"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"}
Sep 06 06:08:00.051039 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.050755    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:08:00.052200 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.052174    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6" event={"ID":"9c49b2e6-4c6a-4cb7-911e-ecbb553b8121","Type":"ContainerStarted","Data":"d2a460645c249d4fd3fd67dd7199ca5361016d9c749cc6a45bba3442179b361c"}
Sep 06 06:08:00.052270 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.052220    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:08:00.069360 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.069298    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" podStartSLOduration=2.210604628 podStartE2EDuration="4.069282954s" totalImagesPullingTime="1.858678326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:08:00.067008694 +0000 UTC m=+833.770561432" watchObservedRunningTime="2026-09-06 06:08:00.069282954 +0000 UTC m=+833.772835690"
Sep 06 06:08:00.085156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:00.085098    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6" podStartSLOduration=2.12767275 podStartE2EDuration="4.085078901s" totalImagesPullingTime="1.957406151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:08:00.084657313 +0000 UTC m=+833.788210049" watchObservedRunningTime="2026-09-06 06:08:00.085078901 +0000 UTC m=+833.788631630"
Sep 06 06:08:06.062107 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:06.062076    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:08:07.032856 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:07.032758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:07.131908 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:07.131870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:11.059885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:11.059846    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8rpm6"
Sep 06 06:08:14.406347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.406304    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"]
Sep 06 06:08:14.412125 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.412095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.416082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.416044    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 06:08:14.416082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.416054    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nbtvj\""
Sep 06 06:08:14.416082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.416051    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 06:08:14.416363 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.416081    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 06 06:08:14.420299 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.420014    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"]
Sep 06 06:08:14.438345 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.438313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:14.578694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.578694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.578919 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nx6x\" (UniqueName: \"kubernetes.io/projected/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-kube-api-access-6nx6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.578919 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.578919 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.579055 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.578927    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nx6x\" (UniqueName: \"kubernetes.io/projected/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-kube-api-access-6nx6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680474    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680844 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680741    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680844 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.680968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.680943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.682783 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.682757    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.683044 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.683026    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.687889 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.687866    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nx6x\" (UniqueName: \"kubernetes.io/projected/54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec-kube-api-access-6nx6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb\" (UID: \"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.723893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.723859    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:14.997498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:14.997471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"]
Sep 06 06:08:14.999749 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:08:14.999722    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ba2c57_489b_4d7f_9aa1_82cf2dcc7cec.slice/crio-6ca32d167a5813a98194cd61129dcb5d6d8948f0f0861092d65d907a66788c12 WatchSource:0}: Error finding container 6ca32d167a5813a98194cd61129dcb5d6d8948f0f0861092d65d907a66788c12: Status 404 returned error can't find the container with id 6ca32d167a5813a98194cd61129dcb5d6d8948f0f0861092d65d907a66788c12
Sep 06 06:08:15.129761 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:15.129729    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb" event={"ID":"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec","Type":"ContainerStarted","Data":"6ca32d167a5813a98194cd61129dcb5d6d8948f0f0861092d65d907a66788c12"}
Sep 06 06:08:17.141779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:17.141743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb" event={"ID":"54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec","Type":"ContainerStarted","Data":"29bc11b68e0719c7b8104f0c4329fe9a4641c16bf8eded76b72470e62a04d952"}
Sep 06 06:08:17.161310 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:17.161243    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb" podStartSLOduration=2.026151031 podStartE2EDuration="3.161222383s" totalImagesPullingTime="1.135071352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:08:17.15879724 +0000 UTC m=+850.862349977" watchObservedRunningTime="2026-09-06 06:08:17.161222383 +0000 UTC m=+850.864775121"
Sep 06 06:08:23.304549 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.304510    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"]
Sep 06 06:08:23.308251 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.308229    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.310931 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.310908    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 06 06:08:23.318016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.317990    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"]
Sep 06 06:08:23.326894 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.326866    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:23.354158 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.354158 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.354375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.354375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.354375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.354375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.354369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bls\" (UniqueName: \"kubernetes.io/projected/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-kube-api-access-p6bls\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.455785 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.455744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.455973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.455805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.455973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.455856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bls\" (UniqueName: \"kubernetes.io/projected/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-kube-api-access-p6bls\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.455973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.455885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.456159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.456008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.456159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.456084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.456279 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.456244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.456396 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.456376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.456483 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.456466    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.458219 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.458194    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.458395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.458378    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.463616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.463593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bls\" (UniqueName: \"kubernetes.io/projected/d8091f7c-9b4d-4ea9-b574-74561e0d2df7-kube-api-access-p6bls\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7xst\" (UID: \"d8091f7c-9b4d-4ea9-b574-74561e0d2df7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.618871 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.618773    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:23.896255 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:23.896230    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"]
Sep 06 06:08:23.897621 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:08:23.897595    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8091f7c_9b4d_4ea9_b574_74561e0d2df7.slice/crio-63ef1d4a2bc7cfea0741aa30aa04f645c028abc4d0e382fcba91af0df7d9d4dc WatchSource:0}: Error finding container 63ef1d4a2bc7cfea0741aa30aa04f645c028abc4d0e382fcba91af0df7d9d4dc: Status 404 returned error can't find the container with id 63ef1d4a2bc7cfea0741aa30aa04f645c028abc4d0e382fcba91af0df7d9d4dc
Sep 06 06:08:24.177655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.177551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst" event={"ID":"d8091f7c-9b4d-4ea9-b574-74561e0d2df7","Type":"ContainerStarted","Data":"f342b84a777d851ecfed6637451a848ec4ccc9cbe40d6fc4446f7bb1133e5e14"}
Sep 06 06:08:24.177655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.177610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst" event={"ID":"d8091f7c-9b4d-4ea9-b574-74561e0d2df7","Type":"ContainerStarted","Data":"63ef1d4a2bc7cfea0741aa30aa04f645c028abc4d0e382fcba91af0df7d9d4dc"}
Sep 06 06:08:24.197239 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.197179    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst" podStartSLOduration=1.026819737 podStartE2EDuration="1.197162724s" totalImagesPullingTime="170.342987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:08:24.195666458 +0000 UTC m=+857.899219194" watchObservedRunningTime="2026-09-06 06:08:24.197162724 +0000 UTC m=+857.900715460"
Sep 06 06:08:24.724014 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.723970    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:24.724503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.724050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:24.736710 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:24.736682    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:25.194222 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:25.194191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb"
Sep 06 06:08:29.921930 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:29.921883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:33.619471 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:33.619420    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:33.619890 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:33.619509    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:33.632153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:33.632124    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:34.239666 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:34.239628    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7xst"
Sep 06 06:08:45.451628 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:45.451597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:45.503619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:45.503583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:08:47.121385 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:08:47.121345    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:00.833839 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:00.833751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:04.507489 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:04.507455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:24.261833 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:24.261767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:24.313648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:24.313610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:33.528248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:33.528212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:34.442610 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:34.442567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:09:34.540767 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:09:34.540726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:33.495275 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:33.495183    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:33.592311 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:33.592270    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:34.615989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:34.615952    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:34.714732 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:34.714698    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:46.395006 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:46.394971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:46.495712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:46.495679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:50.529696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:50.529659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:50.645922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:50.645885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:50.835266 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:50.835221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:51.303841 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:51.303742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:51.947623 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:51.947583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:52.044553 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:52.044521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:52.384197 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:52.384167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:53.548496 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:53.548461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:53.644076 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:53.644025    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:54.787790 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.787755    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:10:54.791163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.791139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:54.793982 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.793961    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ceca3b-serving-cert\""
Sep 06 06:10:54.805349 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.805327    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:10:54.826793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.826756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:54.926923 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.926882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:54.933884 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.933859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:54.934026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.933903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:54.934106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.934024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:54.934106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:54.934073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69slk\" (UniqueName: \"kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.026786 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.026748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:55.034818 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.034784    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.034993 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.034835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.034993 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.034900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.034993 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.034924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69slk\" (UniqueName: \"kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.035571 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.035546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.037451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.037427    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.037640 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.037618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.051127 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.051017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69slk\" (UniqueName: \"kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk\") pod \"maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.103691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.103640    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.218414 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.218327    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:55.381939 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.381913    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:10:55.384650 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:10:55.384610    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b2c4d6_632d_4137_886f_970d10618a66.slice/crio-3655f80e8c10d4c9bbc23e563576a0ddf3d1bb44ebeba92c2d87eacff410a684 WatchSource:0}: Error finding container 3655f80e8c10d4c9bbc23e563576a0ddf3d1bb44ebeba92c2d87eacff410a684: Status 404 returned error can't find the container with id 3655f80e8c10d4c9bbc23e563576a0ddf3d1bb44ebeba92c2d87eacff410a684
Sep 06 06:10:55.386433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.386408    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 06:10:55.413523 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.413488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:55.519268 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.519234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:55.526864 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.526831    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:10:55.531428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.531407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.534347 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.534322    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ceca3b-dockercfg-gtwxd\""
Sep 06 06:10:55.534609 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.534594    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ceca3b\""
Sep 06 06:10:55.536525 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.536504    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:10:55.541100 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.540896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.543778 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.543749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:10:55.554754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.554723    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:10:55.643284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.643201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wtnb\" (UniqueName: \"kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.643284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.643243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.643284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.643270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.643549 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.643462    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8llpn\" (UniqueName: \"kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.744275 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.744234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wtnb\" (UniqueName: \"kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.744275 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.744277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.744475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.744304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.744475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.744397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8llpn\" (UniqueName: \"kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.745230 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.745209    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.745284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.745244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.752684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.752656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wtnb\" (UniqueName: \"kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb\") pod \"payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.752684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.752673    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8llpn\" (UniqueName: \"kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn\") pod \"payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.851154 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.851113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:55.861433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.861410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:55.922134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.922043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" event={"ID":"55b2c4d6-632d-4137-886f-970d10618a66","Type":"ContainerStarted","Data":"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"}
Sep 06 06:10:55.922243 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.922144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" event={"ID":"55b2c4d6-632d-4137-886f-970d10618a66","Type":"ContainerStarted","Data":"3655f80e8c10d4c9bbc23e563576a0ddf3d1bb44ebeba92c2d87eacff410a684"}
Sep 06 06:10:55.923009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.922895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:10:55.943667 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:55.943617    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" podStartSLOduration=1.560288463 podStartE2EDuration="1.943600529s" totalImagesPullingTime="383.312066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:10:55.940567826 +0000 UTC m=+1009.644120602" watchObservedRunningTime="2026-09-06 06:10:55.943600529 +0000 UTC m=+1009.647153265"
Sep 06 06:10:56.128510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.128474    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:10:56.131875 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:10:56.131837    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6567d6_997b_4d57_84b4_74e2126266a5.slice/crio-990eff5b83f324f8b905bef1496adf5d961cec6161d468cedbcd4a7c8fe880e0 WatchSource:0}: Error finding container 990eff5b83f324f8b905bef1496adf5d961cec6161d468cedbcd4a7c8fe880e0: Status 404 returned error can't find the container with id 990eff5b83f324f8b905bef1496adf5d961cec6161d468cedbcd4a7c8fe880e0
Sep 06 06:10:56.144806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.144783    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:10:56.146094 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:10:56.146051    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7ca98f_151f_4e34_8b28_b37e2318126b.slice/crio-68c39d9b725f9d5acd435b1eea869d13cf25474a8467384f527b99f968beca9b WatchSource:0}: Error finding container 68c39d9b725f9d5acd435b1eea869d13cf25474a8467384f527b99f968beca9b: Status 404 returned error can't find the container with id 68c39d9b725f9d5acd435b1eea869d13cf25474a8467384f527b99f968beca9b
Sep 06 06:10:56.930432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.930402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" event={"ID":"8e6567d6-997b-4d57-84b4-74e2126266a5","Type":"ContainerStarted","Data":"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"}
Sep 06 06:10:56.930900 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.930461    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" event={"ID":"8e6567d6-997b-4d57-84b4-74e2126266a5","Type":"ContainerStarted","Data":"990eff5b83f324f8b905bef1496adf5d961cec6161d468cedbcd4a7c8fe880e0"}
Sep 06 06:10:56.930900 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.930496    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:10:56.932420 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.932372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" event={"ID":"5b7ca98f-151f-4e34-8b28-b37e2318126b","Type":"ContainerStarted","Data":"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"}
Sep 06 06:10:56.932564 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.932458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" event={"ID":"5b7ca98f-151f-4e34-8b28-b37e2318126b","Type":"ContainerStarted","Data":"68c39d9b725f9d5acd435b1eea869d13cf25474a8467384f527b99f968beca9b"}
Sep 06 06:10:56.932564 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.932493    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:10:56.947466 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.947415    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" podStartSLOduration=1.947401351 podStartE2EDuration="1.947401351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:10:56.945901085 +0000 UTC m=+1010.649453823" watchObservedRunningTime="2026-09-06 06:10:56.947401351 +0000 UTC m=+1010.650954086"
Sep 06 06:10:56.965586 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:56.965537    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" podStartSLOduration=1.9655175040000001 podStartE2EDuration="1.965517504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:10:56.961588675 +0000 UTC m=+1010.665141415" watchObservedRunningTime="2026-09-06 06:10:56.965517504 +0000 UTC m=+1010.669070242"
Sep 06 06:10:58.276868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:58.276830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:58.377919 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:58.377885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:58.482671 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:58.482640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:58.780670 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:58.780630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:10:59.057978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:10:59.057944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:00.254637 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:00.254602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:00.531378 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:00.531260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:00.907895 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:00.907856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:01.006443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:01.006387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:01.142498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:01.142462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.016272 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.016237    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:02.020228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.020207    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.023015 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.022993    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5da01692-serving-cert\""
Sep 06 06:11:02.034265 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.034240    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:02.059900 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.059851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.163441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.163398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.196894 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.196860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.197082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.196917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.197082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.196944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8g7j\" (UniqueName: \"kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.197082 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.196969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.262889 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.262853    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.297978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.297895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.297978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.297958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.298229 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.297983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8g7j\" (UniqueName: \"kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.298229 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.298009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.298721 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.298700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.300432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.300409    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.300526 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.300468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.305863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.305838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8g7j\" (UniqueName: \"kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j\") pod \"maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.333874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.333848    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:02.607035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.607005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:02.608368 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:11:02.608334    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3064671_9176_441f_a2d8_090323449540.slice/crio-55af3f876eba8ac576ef46594df8ad53bc29ab6f05b4758a23d876c84a4fb193 WatchSource:0}: Error finding container 55af3f876eba8ac576ef46594df8ad53bc29ab6f05b4758a23d876c84a4fb193: Status 404 returned error can't find the container with id 55af3f876eba8ac576ef46594df8ad53bc29ab6f05b4758a23d876c84a4fb193
Sep 06 06:11:02.642675 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.642642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.741344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.741306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:02.945802 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.945716    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:11:02.967118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:02.967093    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" event={"ID":"c3064671-9176-441f-a2d8-090323449540","Type":"ContainerStarted","Data":"55af3f876eba8ac576ef46594df8ad53bc29ab6f05b4758a23d876c84a4fb193"}
Sep 06 06:11:03.564276 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:03.564241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:03.656517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:03.656485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:03.974596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:03.974494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" event={"ID":"c3064671-9176-441f-a2d8-090323449540","Type":"ContainerStarted","Data":"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"}
Sep 06 06:11:03.974596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:03.974571    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:04.010403 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.010343    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" podStartSLOduration=2.65664122 podStartE2EDuration="3.010324704s" totalImagesPullingTime="353.683484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:11:04.009277814 +0000 UTC m=+1017.712830551" watchObservedRunningTime="2026-09-06 06:11:04.010324704 +0000 UTC m=+1017.713877444"
Sep 06 06:11:04.786941 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.786912    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:11:04.789797 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.789781    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:04.793659 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.793622    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-e75768-serving-cert\""
Sep 06 06:11:04.795051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.795023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:11:04.825103 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.825019    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:04.919822 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.919783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:04.919994 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.919830    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:04.919994 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.919972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:04.920147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.920007    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8fkl\" (UniqueName: \"kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:04.925973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:04.925942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:05.021219 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.021180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.021395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.021227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8fkl\" (UniqueName: \"kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.021395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.021278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.021395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.021299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.021586 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:11:05.021413    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-e75768-serving-cert: secret "maas-api-e2e-autores-e75768-serving-cert" not found
Sep 06 06:11:05.021586 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:11:05.021504    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls podName:57703f44-9443-441c-a72b-114e6c7936ef nodeName:}" failed. No retries permitted until 2026-09-06 06:11:05.521483128 +0000 UTC m=+1019.225035858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls") pod "maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" (UID: "57703f44-9443-441c-a72b-114e6c7936ef") : secret "maas-api-e2e-autores-e75768-serving-cert" not found
Sep 06 06:11:05.021999 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.021977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.023707 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.023687    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.031307 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.031276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8fkl\" (UniqueName: \"kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.290661 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.290628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:05.389590 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.389553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:05.440003 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.439965    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:11:05.443581 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.443565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.446218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.446193    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-e75768-dockercfg-htm2j\""
Sep 06 06:11:05.446354 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.446240    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-e75768\""
Sep 06 06:11:05.455392 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.455364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:11:05.526131 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.526044    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.528900 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.528872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") pod \"maas-api-e2e-autores-e75768-65dc576bc8-dwlj6\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.627051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.627014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.627249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.627075    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p9tf\" (UniqueName: \"kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.704621 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.704586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:05.727975 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.727930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.728126 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.727986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p9tf\" (UniqueName: \"kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.728837 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.728803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.735834 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.735807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p9tf\" (UniqueName: \"kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf\") pod \"payload-processing-e2e-autores-e75768-598dc99957-x9npk\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") " pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.757596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.757395    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:05.979246 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:05.979223    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:11:05.980814 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:11:05.980787    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57703f44_9443_441c_a72b_114e6c7936ef.slice/crio-ba5140af53d2046151a19eb2691b3169b3fa5e2a08eeeb10ad272b1bbb579404 WatchSource:0}: Error finding container ba5140af53d2046151a19eb2691b3169b3fa5e2a08eeeb10ad272b1bbb579404: Status 404 returned error can't find the container with id ba5140af53d2046151a19eb2691b3169b3fa5e2a08eeeb10ad272b1bbb579404
Sep 06 06:11:06.048509 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.048479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:11:06.050123 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:11:06.050094    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdae1fd_80ff_4203_9ee8_3f5685de81ad.slice/crio-452dc933b449e328783565b6adfbe657bdb7ee95a87cac5350aa05d2e2ad52b4 WatchSource:0}: Error finding container 452dc933b449e328783565b6adfbe657bdb7ee95a87cac5350aa05d2e2ad52b4: Status 404 returned error can't find the container with id 452dc933b449e328783565b6adfbe657bdb7ee95a87cac5350aa05d2e2ad52b4
Sep 06 06:11:06.994923 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.994862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" event={"ID":"57703f44-9443-441c-a72b-114e6c7936ef","Type":"ContainerStarted","Data":"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"}
Sep 06 06:11:06.995418 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.994974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:06.995418 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.995004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" event={"ID":"57703f44-9443-441c-a72b-114e6c7936ef","Type":"ContainerStarted","Data":"ba5140af53d2046151a19eb2691b3169b3fa5e2a08eeeb10ad272b1bbb579404"}
Sep 06 06:11:06.996462 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.996438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" event={"ID":"5fdae1fd-80ff-4203-9ee8-3f5685de81ad","Type":"ContainerStarted","Data":"09c5f703e0e6f8ef936d4206e7397cac80ab28dd4716ebd3d7678f1105889e48"}
Sep 06 06:11:06.996602 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.996485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" event={"ID":"5fdae1fd-80ff-4203-9ee8-3f5685de81ad","Type":"ContainerStarted","Data":"452dc933b449e328783565b6adfbe657bdb7ee95a87cac5350aa05d2e2ad52b4"}
Sep 06 06:11:06.996602 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:06.996522    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:07.013105 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:07.013039    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" podStartSLOduration=2.684879319 podStartE2EDuration="3.013023927s" totalImagesPullingTime="328.144608ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:11:07.010129441 +0000 UTC m=+1020.713682179" watchObservedRunningTime="2026-09-06 06:11:07.013023927 +0000 UTC m=+1020.716576711"
Sep 06 06:11:07.025178 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:07.024847    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" podStartSLOduration=2.024824238 podStartE2EDuration="2.024824238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:11:07.024529594 +0000 UTC m=+1020.728082334" watchObservedRunningTime="2026-09-06 06:11:07.024824238 +0000 UTC m=+1020.728376975"
Sep 06 06:11:07.941971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:07.941942    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:11:08.947395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:08.947366    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:11:09.399412 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:09.399369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:09.546694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:09.546663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:09.641575 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:09.641539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:09.737868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:09.737771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:09.986166 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:09.986136    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:12.370216 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:12.370179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:12.486675 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:12.486631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:12.537913 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:12.537873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:13.008934 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:13.008897    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:11:13.913119 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:13.913048    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:15.992337 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:15.992304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:16.305941 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:16.305877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:16.406052 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:16.406000    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:16.521371 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:16.521303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:18.005476 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:18.005434    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:11:18.354593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:18.354560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:18.653188 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:18.653097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:18.946435 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:18.946348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:19.216013 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:19.215908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:21.599210 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:21.599170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:21.699466 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:21.699432    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:22.787981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.787948    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:11:22.801926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.801896    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:11:22.802175 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.802112    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.804913 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.804890    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0bce0b-serving-cert\""
Sep 06 06:11:22.822488 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.822443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:22.875512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.875475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.875512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.875513    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.875704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.875555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.875704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.875596    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfmg\" (UniqueName: \"kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.918127 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.918094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:22.976737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.976700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfmg\" (UniqueName: \"kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.976990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.976810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.976990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.976838    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.976990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.976898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.977285 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:11:22.977000    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-serving-cert: secret "maas-api-e2e-inf-b-0bce0b-serving-cert" not found
Sep 06 06:11:22.977285 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:11:22.977087    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls podName:89b42528-4b57-4e46-88bc-81834e8652ce nodeName:}" failed. No retries permitted until 2026-09-06 06:11:23.477043021 +0000 UTC m=+1037.180595748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls") pod "maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" (UID: "89b42528-4b57-4e46-88bc-81834e8652ce") : secret "maas-api-e2e-inf-b-0bce0b-serving-cert" not found
Sep 06 06:11:22.977621 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.977597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.979498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.979476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:22.985085 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:22.985049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfmg\" (UniqueName: \"kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:23.167864 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.167831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.390447 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.390416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.480130 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.480013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:23.482596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.482567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") pod \"maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:23.489971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.489925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.715224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.715190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:23.785627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.785592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.887534 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.887498    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.985293 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.985258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:23.994650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:23.994625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:11:23.997149 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:11:23.997121    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b42528_4b57_4e46_88bc_81834e8652ce.slice/crio-d42eb1fccbe9e49a27d22435e62b6702a9ee3b97cdee8a71bd73d0e2c6064df8 WatchSource:0}: Error finding container d42eb1fccbe9e49a27d22435e62b6702a9ee3b97cdee8a71bd73d0e2c6064df8: Status 404 returned error can't find the container with id d42eb1fccbe9e49a27d22435e62b6702a9ee3b97cdee8a71bd73d0e2c6064df8
Sep 06 06:11:24.093139 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:24.093103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" event={"ID":"89b42528-4b57-4e46-88bc-81834e8652ce","Type":"ContainerStarted","Data":"d42eb1fccbe9e49a27d22435e62b6702a9ee3b97cdee8a71bd73d0e2c6064df8"}
Sep 06 06:11:24.104910 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:24.104863    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:24.691218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:24.691125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:25.100656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:25.100614    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" event={"ID":"89b42528-4b57-4e46-88bc-81834e8652ce","Type":"ContainerStarted","Data":"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"}
Sep 06 06:11:25.101041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:25.100670    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:25.119160 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:25.119106    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" podStartSLOduration=2.752479039 podStartE2EDuration="3.119089984s" totalImagesPullingTime="366.610945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:11:25.117789046 +0000 UTC m=+1038.821341783" watchObservedRunningTime="2026-09-06 06:11:25.119089984 +0000 UTC m=+1038.822642719"
Sep 06 06:11:25.353091 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:25.352990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:26.111102 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:26.111043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:26.914454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:26.914418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:27.644847 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:27.644809    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:28.392100 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:28.392051    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:29.927616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:29.927585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:29.984920 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:29.984881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:31.111592 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:31.111561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:11:42.309613 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:42.309576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:42.849800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:42.849764    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:42.850030 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:42.850006    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" podUID="c3064671-9176-441f-a2d8-090323449540" containerName="maas-api" containerID="cri-o://ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35" gracePeriod=30
Sep 06 06:11:42.878922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:42.878895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:42.946767 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:42.946735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.048672 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.048626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.098669 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.098647    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:43.106264 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.106202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.196969 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.196928    2567 generic.go:356] "Generic (PLEG): container finished" podID="c3064671-9176-441f-a2d8-090323449540" containerID="ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35" exitCode=0
Sep 06 06:11:43.197163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.197026    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"
Sep 06 06:11:43.197163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.197028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" event={"ID":"c3064671-9176-441f-a2d8-090323449540","Type":"ContainerDied","Data":"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"}
Sep 06 06:11:43.197163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.197127    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx" event={"ID":"c3064671-9176-441f-a2d8-090323449540","Type":"ContainerDied","Data":"55af3f876eba8ac576ef46594df8ad53bc29ab6f05b4758a23d876c84a4fb193"}
Sep 06 06:11:43.197163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.197155    2567 scope.go:117] "RemoveContainer" containerID="ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"
Sep 06 06:11:43.209520 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.209493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.209975 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.209948    2567 scope.go:117] "RemoveContainer" containerID="ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"
Sep 06 06:11:43.210314 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:11:43.210276    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35\": container with ID starting with ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35 not found: ID does not exist" containerID="ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"
Sep 06 06:11:43.210314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.210305    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35"} err="failed to get container status \"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35\": rpc error: code = NotFound desc = could not find container \"ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35\": container with ID starting with ec2c8b4b400981db59b74c0fff2f2854a70b055d0592ee00080406d2f3db0a35 not found: ID does not exist"
Sep 06 06:11:43.258078 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.258040    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls\") pod \"c3064671-9176-441f-a2d8-090323449540\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") "
Sep 06 06:11:43.258247 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.258109    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8g7j\" (UniqueName: \"kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j\") pod \"c3064671-9176-441f-a2d8-090323449540\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") "
Sep 06 06:11:43.258247 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.258195    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs\") pod \"c3064671-9176-441f-a2d8-090323449540\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") "
Sep 06 06:11:43.258247 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.258233    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca\") pod \"c3064671-9176-441f-a2d8-090323449540\" (UID: \"c3064671-9176-441f-a2d8-090323449540\") "
Sep 06 06:11:43.258929 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.258901    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3064671-9176-441f-a2d8-090323449540" (UID: "c3064671-9176-441f-a2d8-090323449540"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:11:43.260572 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.260540    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c3064671-9176-441f-a2d8-090323449540" (UID: "c3064671-9176-441f-a2d8-090323449540"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:11:43.260572 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.260558    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3064671-9176-441f-a2d8-090323449540" (UID: "c3064671-9176-441f-a2d8-090323449540"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:11:43.260695 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.260620    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j" (OuterVolumeSpecName: "kube-api-access-l8g7j") pod "c3064671-9176-441f-a2d8-090323449540" (UID: "c3064671-9176-441f-a2d8-090323449540"). InnerVolumeSpecName "kube-api-access-l8g7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:11:43.359751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.359671    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:11:43.359751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.359700    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8g7j\" (UniqueName: \"kubernetes.io/projected/c3064671-9176-441f-a2d8-090323449540-kube-api-access-l8g7j\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:11:43.359751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.359709    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3064671-9176-441f-a2d8-090323449540-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:11:43.359751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.359717    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3064671-9176-441f-a2d8-090323449540-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:11:43.518669 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.518633    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:43.522232 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.522197    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5da01692-d8b4f6bfc-fq9kx"]
Sep 06 06:11:43.685514 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.685421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.790518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.790471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.886053 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.886013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:43.985421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:43.985331    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:44.148119 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:44.148085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:44.318250 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:44.318214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:44.786576 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:44.786500    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3064671-9176-441f-a2d8-090323449540" path="/var/lib/kubelet/pods/c3064671-9176-441f-a2d8-090323449540/volumes"
Sep 06 06:11:46.417522 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.417486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:46.515168 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.515109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:46.731838 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.731747    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:11:46.732866 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.732838    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3064671-9176-441f-a2d8-090323449540" containerName="maas-api"
Sep 06 06:11:46.733004 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.732994    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3064671-9176-441f-a2d8-090323449540" containerName="maas-api"
Sep 06 06:11:46.733324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.733310    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3064671-9176-441f-a2d8-090323449540" containerName="maas-api"
Sep 06 06:11:46.739738 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.739712    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.743516 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.743497    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-06e64f94-serving-cert\""
Sep 06 06:11:46.754322 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.754294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:11:46.821018 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.820987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:46.891253 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.891216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.891253 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.891252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxxw\" (UniqueName: \"kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.891455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.891297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.891455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.891352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.908938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.908899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:46.992280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.992199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.992280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.992232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxxw\" (UniqueName: \"kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.992280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.992265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.992514 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.992306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.993104 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.993083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.994808 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.994782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:46.994808 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:46.994800    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:47.000103 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.000081    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxxw\" (UniqueName: \"kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw\") pod \"maas-api-e2e-migrate-06e64f94-54f695595b-wffbw\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:47.008844 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.008817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:47.050244 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.050212    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:47.286567 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.286480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:47.324853 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.324821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:11:47.326288 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:11:47.326257    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3282b393_d169_47e0_a2db_152e34ea4dfd.slice/crio-579e70519e42d4808a6266fbad7a9b9b5c97dad44089a6a86ca70e2f9f7ce851 WatchSource:0}: Error finding container 579e70519e42d4808a6266fbad7a9b9b5c97dad44089a6a86ca70e2f9f7ce851: Status 404 returned error can't find the container with id 579e70519e42d4808a6266fbad7a9b9b5c97dad44089a6a86ca70e2f9f7ce851
Sep 06 06:11:47.386859 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:47.386826    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:11:48.227768 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:48.227730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" event={"ID":"3282b393-d169-47e0-a2db-152e34ea4dfd","Type":"ContainerStarted","Data":"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"}
Sep 06 06:11:48.228170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:48.227819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" event={"ID":"3282b393-d169-47e0-a2db-152e34ea4dfd","Type":"ContainerStarted","Data":"579e70519e42d4808a6266fbad7a9b9b5c97dad44089a6a86ca70e2f9f7ce851"}
Sep 06 06:11:48.228170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:48.227841    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:11:48.246475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:48.246427    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" podStartSLOduration=1.9420576189999998 podStartE2EDuration="2.246406553s" totalImagesPullingTime="304.348934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:11:48.243688255 +0000 UTC m=+1061.947240991" watchObservedRunningTime="2026-09-06 06:11:48.246406553 +0000 UTC m=+1061.949959297"
Sep 06 06:11:54.240504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:11:54.240472    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:12:09.914217 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:09.914181    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:17.923239 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:17.923189    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:12:17.924036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:17.924007    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" podUID="3282b393-d169-47e0-a2db-152e34ea4dfd" containerName="maas-api" containerID="cri-o://03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168" gracePeriod=30
Sep 06 06:12:17.950225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:17.950191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.021611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.021573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.088691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.088647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.183766 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.183704    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:12:18.185909 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.185884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.279538 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.279505    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs\") pod \"3282b393-d169-47e0-a2db-152e34ea4dfd\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") "
Sep 06 06:12:18.279714 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.279550    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls\") pod \"3282b393-d169-47e0-a2db-152e34ea4dfd\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") "
Sep 06 06:12:18.279714 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.279593    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca\") pod \"3282b393-d169-47e0-a2db-152e34ea4dfd\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") "
Sep 06 06:12:18.279714 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.279619    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pxxw\" (UniqueName: \"kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw\") pod \"3282b393-d169-47e0-a2db-152e34ea4dfd\" (UID: \"3282b393-d169-47e0-a2db-152e34ea4dfd\") "
Sep 06 06:12:18.279981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.279957    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3282b393-d169-47e0-a2db-152e34ea4dfd" (UID: "3282b393-d169-47e0-a2db-152e34ea4dfd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:12:18.281806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.281782    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw" (OuterVolumeSpecName: "kube-api-access-4pxxw") pod "3282b393-d169-47e0-a2db-152e34ea4dfd" (UID: "3282b393-d169-47e0-a2db-152e34ea4dfd"). InnerVolumeSpecName "kube-api-access-4pxxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:12:18.282223 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.282196    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3282b393-d169-47e0-a2db-152e34ea4dfd" (UID: "3282b393-d169-47e0-a2db-152e34ea4dfd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:12:18.282321 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.282223    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3282b393-d169-47e0-a2db-152e34ea4dfd" (UID: "3282b393-d169-47e0-a2db-152e34ea4dfd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:12:18.286951 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.286921    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.381248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.381215    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:18.381248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.381246    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3282b393-d169-47e0-a2db-152e34ea4dfd-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:18.381514 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.381258    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3282b393-d169-47e0-a2db-152e34ea4dfd-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:18.381514 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.381269    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pxxw\" (UniqueName: \"kubernetes.io/projected/3282b393-d169-47e0-a2db-152e34ea4dfd-kube-api-access-4pxxw\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:18.387015 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.386985    2567 generic.go:356] "Generic (PLEG): container finished" podID="3282b393-d169-47e0-a2db-152e34ea4dfd" containerID="03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168" exitCode=0
Sep 06 06:12:18.387189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.387047    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"
Sep 06 06:12:18.387189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.387080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" event={"ID":"3282b393-d169-47e0-a2db-152e34ea4dfd","Type":"ContainerDied","Data":"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"}
Sep 06 06:12:18.387189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.387136    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw" event={"ID":"3282b393-d169-47e0-a2db-152e34ea4dfd","Type":"ContainerDied","Data":"579e70519e42d4808a6266fbad7a9b9b5c97dad44089a6a86ca70e2f9f7ce851"}
Sep 06 06:12:18.387189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.387158    2567 scope.go:117] "RemoveContainer" containerID="03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"
Sep 06 06:12:18.398712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.398688    2567 scope.go:117] "RemoveContainer" containerID="03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"
Sep 06 06:12:18.399012 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:18.398975    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168\": container with ID starting with 03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168 not found: ID does not exist" containerID="03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"
Sep 06 06:12:18.399100 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.399019    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168"} err="failed to get container status \"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168\": rpc error: code = NotFound desc = could not find container \"03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168\": container with ID starting with 03029ee0d7e59315514d17558bb4f838a7158ef21124e5e537d5a67c689fe168 not found: ID does not exist"
Sep 06 06:12:18.411612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.411583    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:12:18.413691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.413671    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-06e64f94-54f695595b-wffbw"]
Sep 06 06:12:18.706970 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.706930    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:18.791115 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:18.787636    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3282b393-d169-47e0-a2db-152e34ea4dfd" path="/var/lib/kubelet/pods/3282b393-d169-47e0-a2db-152e34ea4dfd/volumes"
Sep 06 06:12:19.005962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:19.005864    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:19.104921 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:19.104880    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:21.434350 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:21.434305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:21.615122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:21.615080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:22.548870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.548831    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:22.549423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.549310    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3282b393-d169-47e0-a2db-152e34ea4dfd" containerName="maas-api"
Sep 06 06:12:22.549423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.549327    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3282b393-d169-47e0-a2db-152e34ea4dfd" containerName="maas-api"
Sep 06 06:12:22.549423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.549384    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3282b393-d169-47e0-a2db-152e34ea4dfd" containerName="maas-api"
Sep 06 06:12:22.553899 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.553880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.556942 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.556918    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-efdea9ac-serving-cert\""
Sep 06 06:12:22.563456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.563158    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:22.582236 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.582207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:22.620231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.620190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.620391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.620305    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6lf8\" (UniqueName: \"kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.620391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.620341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.620481 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.620409    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.683615 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.683572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:22.721631 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.721594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.721779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.721661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6lf8\" (UniqueName: \"kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.721779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.721683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.721779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.721711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.721779 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:22.721771    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-serving-cert: secret "maas-api-e2e-ait-efdea9ac-serving-cert" not found
Sep 06 06:12:22.721997 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:22.721866    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls podName:3135163b-2f9a-48da-a550-f7330ac6d7f5 nodeName:}" failed. No retries permitted until 2026-09-06 06:12:23.221847744 +0000 UTC m=+1096.925400458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls") pod "maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" (UID: "3135163b-2f9a-48da-a550-f7330ac6d7f5") : secret "maas-api-e2e-ait-efdea9ac-serving-cert" not found
Sep 06 06:12:22.722480 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.722463    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.724534 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.724510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.730570 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.730543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6lf8\" (UniqueName: \"kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:22.782793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:22.782758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:23.060484 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.060447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:23.161257 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.161213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:23.227000 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.226963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:23.229497 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.229463    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") pod \"maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:23.467115 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.467002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:23.742615 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:23.742580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:23.743994 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:12:23.743959    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3135163b_2f9a_48da_a550_f7330ac6d7f5.slice/crio-d948faf431ac31f6151e8a9ee3d8b1d9a9eb57f7c8c95e76d1e85a4200bb5753 WatchSource:0}: Error finding container d948faf431ac31f6151e8a9ee3d8b1d9a9eb57f7c8c95e76d1e85a4200bb5753: Status 404 returned error can't find the container with id d948faf431ac31f6151e8a9ee3d8b1d9a9eb57f7c8c95e76d1e85a4200bb5753
Sep 06 06:12:24.420293 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:24.420206    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" event={"ID":"3135163b-2f9a-48da-a550-f7330ac6d7f5","Type":"ContainerStarted","Data":"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"}
Sep 06 06:12:24.420293 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:24.420279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" event={"ID":"3135163b-2f9a-48da-a550-f7330ac6d7f5","Type":"ContainerStarted","Data":"d948faf431ac31f6151e8a9ee3d8b1d9a9eb57f7c8c95e76d1e85a4200bb5753"}
Sep 06 06:12:24.420480 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:24.420326    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:24.439948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:24.439885    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" podStartSLOduration=2.062389763 podStartE2EDuration="2.439866379s" totalImagesPullingTime="377.476616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:12:24.438310684 +0000 UTC m=+1098.141863419" watchObservedRunningTime="2026-09-06 06:12:24.439866379 +0000 UTC m=+1098.143419117"
Sep 06 06:12:30.434106 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:30.434051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:47.716626 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.716438    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:12:47.721296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.721269    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.724868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.724839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:12:47.725588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.725362    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-e75768\"/\"kube-root-ca.crt\""
Sep 06 06:12:47.725588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.725414    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-e75768\"/\"default-dockercfg-zbncr\""
Sep 06 06:12:47.725588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.725364    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-e75768\"/\"autores-model-e75768-kserve-self-signed-certs\""
Sep 06 06:12:47.725588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.725362    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-e75768\"/\"openshift-service-ca.crt\""
Sep 06 06:12:47.752297 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.752439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.752439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8tsf\" (UniqueName: \"kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.752439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.752439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.752596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.752518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.801158 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.801111    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:47.852924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.852885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.852976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853011    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8tsf\" (UniqueName: \"kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853842 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.853968 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.853921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.855839 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.855816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.855924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.855872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:47.861108 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:47.861055    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8tsf\" (UniqueName: \"kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf\") pod \"autores-model-e75768-kserve-7f8d56f94c-nbpbf\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") " pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:48.032730 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.032639    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:48.311513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.311486    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:12:48.313192 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:12:48.313163    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5df396d_aa84_46cc_a025_98cc2e35992b.slice/crio-ed3c24780cb121e4b86efc0653dc80f887ee206ec78e7944d2d74fae2be74d19 WatchSource:0}: Error finding container ed3c24780cb121e4b86efc0653dc80f887ee206ec78e7944d2d74fae2be74d19: Status 404 returned error can't find the container with id ed3c24780cb121e4b86efc0653dc80f887ee206ec78e7944d2d74fae2be74d19
Sep 06 06:12:48.442444 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.442408    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:48.550024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.549987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" event={"ID":"d5df396d-aa84-46cc-a025-98cc2e35992b","Type":"ContainerStarted","Data":"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"}
Sep 06 06:12:48.550248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.550040    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" event={"ID":"d5df396d-aa84-46cc-a025-98cc2e35992b","Type":"ContainerStarted","Data":"ed3c24780cb121e4b86efc0653dc80f887ee206ec78e7944d2d74fae2be74d19"}
Sep 06 06:12:48.568648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:48.568302    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" podStartSLOduration=1.568285964 podStartE2EDuration="1.568285964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:12:48.565968297 +0000 UTC m=+1122.269521034" watchObservedRunningTime="2026-09-06 06:12:48.568285964 +0000 UTC m=+1122.271838691"
Sep 06 06:12:53.214859 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.214824    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:53.215263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.215020    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" podUID="3135163b-2f9a-48da-a550-f7330ac6d7f5" containerName="maas-api" containerID="cri-o://a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793" gracePeriod=30
Sep 06 06:12:53.238520 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.238485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:53.307812 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.307776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:53.373293 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.373258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:53.448390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.448353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:53.458308 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.458286    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:53.506588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.506502    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6lf8\" (UniqueName: \"kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8\") pod \"3135163b-2f9a-48da-a550-f7330ac6d7f5\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") "
Sep 06 06:12:53.506588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.506541    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs\") pod \"3135163b-2f9a-48da-a550-f7330ac6d7f5\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") "
Sep 06 06:12:53.506800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.506634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") pod \"3135163b-2f9a-48da-a550-f7330ac6d7f5\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") "
Sep 06 06:12:53.506800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.506667    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca\") pod \"3135163b-2f9a-48da-a550-f7330ac6d7f5\" (UID: \"3135163b-2f9a-48da-a550-f7330ac6d7f5\") "
Sep 06 06:12:53.507117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.507090    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3135163b-2f9a-48da-a550-f7330ac6d7f5" (UID: "3135163b-2f9a-48da-a550-f7330ac6d7f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:12:53.508726 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.508703    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3135163b-2f9a-48da-a550-f7330ac6d7f5" (UID: "3135163b-2f9a-48da-a550-f7330ac6d7f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:12:53.508806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.508735    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3135163b-2f9a-48da-a550-f7330ac6d7f5" (UID: "3135163b-2f9a-48da-a550-f7330ac6d7f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:12:53.508806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.508771    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8" (OuterVolumeSpecName: "kube-api-access-d6lf8") pod "3135163b-2f9a-48da-a550-f7330ac6d7f5" (UID: "3135163b-2f9a-48da-a550-f7330ac6d7f5"). InnerVolumeSpecName "kube-api-access-d6lf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:12:53.510954 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.510911    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:53.578593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.578564    2567 generic.go:356] "Generic (PLEG): container finished" podID="3135163b-2f9a-48da-a550-f7330ac6d7f5" containerID="a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793" exitCode=0
Sep 06 06:12:53.578731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.578607    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"
Sep 06 06:12:53.578731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.578651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" event={"ID":"3135163b-2f9a-48da-a550-f7330ac6d7f5","Type":"ContainerDied","Data":"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"}
Sep 06 06:12:53.578731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.578713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns" event={"ID":"3135163b-2f9a-48da-a550-f7330ac6d7f5","Type":"ContainerDied","Data":"d948faf431ac31f6151e8a9ee3d8b1d9a9eb57f7c8c95e76d1e85a4200bb5753"}
Sep 06 06:12:53.578854 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.578749    2567 scope.go:117] "RemoveContainer" containerID="a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"
Sep 06 06:12:53.593311 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.593291    2567 scope.go:117] "RemoveContainer" containerID="a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"
Sep 06 06:12:53.593631 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:53.593610    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793\": container with ID starting with a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793 not found: ID does not exist" containerID="a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"
Sep 06 06:12:53.593700 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.593645    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793"} err="failed to get container status \"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793\": rpc error: code = NotFound desc = could not find container \"a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793\": container with ID starting with a26323a1bcbb8b2391cda366fd88cd69cc9251bf169317eab42a8bd36655c793 not found: ID does not exist"
Sep 06 06:12:53.605510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.605476    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:53.608277 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.608255    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6lf8\" (UniqueName: \"kubernetes.io/projected/3135163b-2f9a-48da-a550-f7330ac6d7f5-kube-api-access-d6lf8\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:53.608400 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.608280    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:53.608400 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.608294    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3135163b-2f9a-48da-a550-f7330ac6d7f5-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:53.608400 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.608306    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3135163b-2f9a-48da-a550-f7330ac6d7f5-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:12:53.608669 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:53.608642    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efdea9ac-65b4bd6cd7-rnsns"]
Sep 06 06:12:54.555523 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:54.555488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:54.786573 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:54.786540    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3135163b-2f9a-48da-a550-f7330ac6d7f5" path="/var/lib/kubelet/pods/3135163b-2f9a-48da-a550-f7330ac6d7f5/volumes"
Sep 06 06:12:54.960456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:54.960349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:55.251974 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:55.251863    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:55.435835 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:55.435795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:57.663937 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:57.663904    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:57.763387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:57.763351    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:58.033183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.033099    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:58.033183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.033155    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:58.051595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.051560    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:58.618238 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.618212    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:12:58.790525 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.790489    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:12:58.791117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.791097    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3135163b-2f9a-48da-a550-f7330ac6d7f5" containerName="maas-api"
Sep 06 06:12:58.791183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.791120    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3135163b-2f9a-48da-a550-f7330ac6d7f5" containerName="maas-api"
Sep 06 06:12:58.791248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.791237    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3135163b-2f9a-48da-a550-f7330ac6d7f5" containerName="maas-api"
Sep 06 06:12:58.796076 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.796036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.798540 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.798517    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f260fc83-serving-cert\""
Sep 06 06:12:58.804371 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.804341    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:12:58.834248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.834213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:58.852384 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.852352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.852384 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.852391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.852615 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.852412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z98ph\" (UniqueName: \"kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.852615 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.852499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.934600 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.934515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:58.953986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.953951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.953986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.953986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.954224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.954006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ph\" (UniqueName: \"kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.954224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.954039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.954224 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:58.954159    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-serving-cert: secret "maas-api-e2e-derive-f260fc83-serving-cert" not found
Sep 06 06:12:58.954224 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:12:58.954218    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls podName:76c2543c-0215-44b4-b359-64b772d3aa1d nodeName:}" failed. No retries permitted until 2026-09-06 06:12:59.454199088 +0000 UTC m=+1133.157751805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls") pod "maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" (UID: "76c2543c-0215-44b4-b359-64b772d3aa1d") : secret "maas-api-e2e-derive-f260fc83-serving-cert" not found
Sep 06 06:12:58.954779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.954759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.956558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.956538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:58.962746 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:58.962720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z98ph\" (UniqueName: \"kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:59.241307 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.241218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:59.340930 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.340891    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:59.411126 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.411088    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:12:59.459300 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.459261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:59.461829 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.461801    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") pod \"maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:59.712517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.712476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:12:59.987670 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:12:59.987639    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:12:59.988637 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:12:59.988612    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c2543c_0215_44b4_b359_64b772d3aa1d.slice/crio-8629d23e78dafb4756c3b1f1f3243fb3f5b3ab0b69a7e013e494f9fd92cc4932 WatchSource:0}: Error finding container 8629d23e78dafb4756c3b1f1f3243fb3f5b3ab0b69a7e013e494f9fd92cc4932: Status 404 returned error can't find the container with id 8629d23e78dafb4756c3b1f1f3243fb3f5b3ab0b69a7e013e494f9fd92cc4932
Sep 06 06:13:00.619451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:00.619369    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" event={"ID":"76c2543c-0215-44b4-b359-64b772d3aa1d","Type":"ContainerStarted","Data":"4466ea3b08984cecba80a02356cc828b7515c6c0de0d175275cb7ec6ea18c3bb"}
Sep 06 06:13:00.619451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:00.619418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" event={"ID":"76c2543c-0215-44b4-b359-64b772d3aa1d","Type":"ContainerStarted","Data":"8629d23e78dafb4756c3b1f1f3243fb3f5b3ab0b69a7e013e494f9fd92cc4932"}
Sep 06 06:13:00.619633 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:00.619498    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:13:00.636192 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:00.636139    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" podStartSLOduration=2.25672301 podStartE2EDuration="2.636124146s" totalImagesPullingTime="379.401136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:13:00.634646921 +0000 UTC m=+1134.338199658" watchObservedRunningTime="2026-09-06 06:13:00.636124146 +0000 UTC m=+1134.339676882"
Sep 06 06:13:06.630026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:06.629992    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:13:12.116477 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:12.116429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:15.311231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:15.311195    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:16.662253 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:16.662213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:17.326483 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:17.326445    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:13:17.345375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:17.345346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:17.678636 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:17.678553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:18.795595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:18.795563    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:19.714985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:19.714947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:27.608510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:27.608471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.445029 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.444995    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:13:28.445336 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.445311    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" podUID="57703f44-9443-441c-a72b-114e6c7936ef" containerName="maas-api" containerID="cri-o://42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566" gracePeriod=30
Sep 06 06:13:28.461432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.461318    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.505261 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.505226    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:13:28.505487 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.505441    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" podUID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" containerName="payload-processing" containerID="cri-o://09c5f703e0e6f8ef936d4206e7397cac80ab28dd4716ebd3d7678f1105889e48" gracePeriod=30
Sep 06 06:13:28.537978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.537948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.601690 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.601635    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.660369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.660340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.718182 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.718146    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:28.762998 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.762971    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:13:28.778972 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.778941    2567 generic.go:356] "Generic (PLEG): container finished" podID="57703f44-9443-441c-a72b-114e6c7936ef" containerID="42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566" exitCode=0
Sep 06 06:13:28.779144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.779118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" event={"ID":"57703f44-9443-441c-a72b-114e6c7936ef","Type":"ContainerDied","Data":"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"}
Sep 06 06:13:28.779324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.779151    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"
Sep 06 06:13:28.779324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.779181    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6" event={"ID":"57703f44-9443-441c-a72b-114e6c7936ef","Type":"ContainerDied","Data":"ba5140af53d2046151a19eb2691b3169b3fa5e2a08eeeb10ad272b1bbb579404"}
Sep 06 06:13:28.779324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.779208    2567 scope.go:117] "RemoveContainer" containerID="42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"
Sep 06 06:13:28.783704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.783656    2567 generic.go:356] "Generic (PLEG): container finished" podID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" containerID="09c5f703e0e6f8ef936d4206e7397cac80ab28dd4716ebd3d7678f1105889e48" exitCode=0
Sep 06 06:13:28.793657 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.793627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" event={"ID":"5fdae1fd-80ff-4203-9ee8-3f5685de81ad","Type":"ContainerDied","Data":"09c5f703e0e6f8ef936d4206e7397cac80ab28dd4716ebd3d7678f1105889e48"}
Sep 06 06:13:28.799161 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.799135    2567 scope.go:117] "RemoveContainer" containerID="42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"
Sep 06 06:13:28.799475 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:13:28.799454    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566\": container with ID starting with 42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566 not found: ID does not exist" containerID="42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"
Sep 06 06:13:28.799523 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.799488    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566"} err="failed to get container status \"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566\": rpc error: code = NotFound desc = could not find container \"42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566\": container with ID starting with 42b3a4916735b43a97859e4ea5f0363a85949176e706ffb48889bfa86bdec566 not found: ID does not exist"
Sep 06 06:13:28.801922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.801901    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:13:28.949519 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs\") pod \"57703f44-9443-441c-a72b-114e6c7936ef\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") "
Sep 06 06:13:28.949762 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949527    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8fkl\" (UniqueName: \"kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl\") pod \"57703f44-9443-441c-a72b-114e6c7936ef\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") "
Sep 06 06:13:28.949762 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949598    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca\") pod \"57703f44-9443-441c-a72b-114e6c7936ef\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") "
Sep 06 06:13:28.949762 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949622    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume\") pod \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") "
Sep 06 06:13:28.949762 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949658    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") pod \"57703f44-9443-441c-a72b-114e6c7936ef\" (UID: \"57703f44-9443-441c-a72b-114e6c7936ef\") "
Sep 06 06:13:28.949762 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949686    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p9tf\" (UniqueName: \"kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf\") pod \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\" (UID: \"5fdae1fd-80ff-4203-9ee8-3f5685de81ad\") "
Sep 06 06:13:28.950038 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.949997    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57703f44-9443-441c-a72b-114e6c7936ef" (UID: "57703f44-9443-441c-a72b-114e6c7936ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:13:28.950227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.950182    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5fdae1fd-80ff-4203-9ee8-3f5685de81ad" (UID: "5fdae1fd-80ff-4203-9ee8-3f5685de81ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:13:28.951971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.951945    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf" (OuterVolumeSpecName: "kube-api-access-8p9tf") pod "5fdae1fd-80ff-4203-9ee8-3f5685de81ad" (UID: "5fdae1fd-80ff-4203-9ee8-3f5685de81ad"). InnerVolumeSpecName "kube-api-access-8p9tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:13:28.952086 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.951949    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57703f44-9443-441c-a72b-114e6c7936ef" (UID: "57703f44-9443-441c-a72b-114e6c7936ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:13:28.952280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.952255    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl" (OuterVolumeSpecName: "kube-api-access-l8fkl") pod "57703f44-9443-441c-a72b-114e6c7936ef" (UID: "57703f44-9443-441c-a72b-114e6c7936ef"). InnerVolumeSpecName "kube-api-access-l8fkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:13:28.952358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:28.952278    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57703f44-9443-441c-a72b-114e6c7936ef" (UID: "57703f44-9443-441c-a72b-114e6c7936ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:13:29.009757 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.009674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.050988    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57703f44-9443-441c-a72b-114e6c7936ef-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.051021    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.051033    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.051042    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p9tf\" (UniqueName: \"kubernetes.io/projected/5fdae1fd-80ff-4203-9ee8-3f5685de81ad-kube-api-access-8p9tf\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.051051    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57703f44-9443-441c-a72b-114e6c7936ef-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.051136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.051078    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8fkl\" (UniqueName: \"kubernetes.io/projected/57703f44-9443-441c-a72b-114e6c7936ef-kube-api-access-l8fkl\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:29.102868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.102828    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:13:29.106033 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.106009    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e75768-65dc576bc8-dwlj6"]
Sep 06 06:13:29.120555 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.120523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.648118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.648086    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:13:29.648397 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.648362    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" podUID="76c2543c-0215-44b4-b359-64b772d3aa1d" containerName="maas-api" containerID="cri-o://4466ea3b08984cecba80a02356cc828b7515c6c0de0d175275cb7ec6ea18c3bb" gracePeriod=30
Sep 06 06:13:29.671266 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.671227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.736820 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.736670    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.792819 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.792792    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"
Sep 06 06:13:29.792991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.792828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk" event={"ID":"5fdae1fd-80ff-4203-9ee8-3f5685de81ad","Type":"ContainerDied","Data":"452dc933b449e328783565b6adfbe657bdb7ee95a87cac5350aa05d2e2ad52b4"}
Sep 06 06:13:29.792991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.792894    2567 scope.go:117] "RemoveContainer" containerID="09c5f703e0e6f8ef936d4206e7397cac80ab28dd4716ebd3d7678f1105889e48"
Sep 06 06:13:29.794611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.794585    2567 generic.go:356] "Generic (PLEG): container finished" podID="76c2543c-0215-44b4-b359-64b772d3aa1d" containerID="4466ea3b08984cecba80a02356cc828b7515c6c0de0d175275cb7ec6ea18c3bb" exitCode=0
Sep 06 06:13:29.794738 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.794632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" event={"ID":"76c2543c-0215-44b4-b359-64b772d3aa1d","Type":"ContainerDied","Data":"4466ea3b08984cecba80a02356cc828b7515c6c0de0d175275cb7ec6ea18c3bb"}
Sep 06 06:13:29.817547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.817451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:13:29.819406 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.819370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.822301 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.822279    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e75768-598dc99957-x9npk"]
Sep 06 06:13:29.872610 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.872571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.897917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.897894    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:13:29.934258 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.934222    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:29.959421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.959382    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca\") pod \"76c2543c-0215-44b4-b359-64b772d3aa1d\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") "
Sep 06 06:13:29.959612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.959457    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") pod \"76c2543c-0215-44b4-b359-64b772d3aa1d\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") "
Sep 06 06:13:29.959612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.959532    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z98ph\" (UniqueName: \"kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph\") pod \"76c2543c-0215-44b4-b359-64b772d3aa1d\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") "
Sep 06 06:13:29.959612 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.959582    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs\") pod \"76c2543c-0215-44b4-b359-64b772d3aa1d\" (UID: \"76c2543c-0215-44b4-b359-64b772d3aa1d\") "
Sep 06 06:13:29.959803 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.959775    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76c2543c-0215-44b4-b359-64b772d3aa1d" (UID: "76c2543c-0215-44b4-b359-64b772d3aa1d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:13:29.961656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.961628    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "76c2543c-0215-44b4-b359-64b772d3aa1d" (UID: "76c2543c-0215-44b4-b359-64b772d3aa1d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:13:29.961770 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.961689    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76c2543c-0215-44b4-b359-64b772d3aa1d" (UID: "76c2543c-0215-44b4-b359-64b772d3aa1d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:13:29.961818 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:29.961795    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph" (OuterVolumeSpecName: "kube-api-access-z98ph") pod "76c2543c-0215-44b4-b359-64b772d3aa1d" (UID: "76c2543c-0215-44b4-b359-64b772d3aa1d"). InnerVolumeSpecName "kube-api-access-z98ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:13:30.016272 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.016194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:30.060390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.060356    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76c2543c-0215-44b4-b359-64b772d3aa1d-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:30.060390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.060383    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:30.060390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.060393    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z98ph\" (UniqueName: \"kubernetes.io/projected/76c2543c-0215-44b4-b359-64b772d3aa1d-kube-api-access-z98ph\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:30.060608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.060402    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76c2543c-0215-44b4-b359-64b772d3aa1d-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:30.659581 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.659551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:30.761456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.761410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:30.786987 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.786957    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57703f44-9443-441c-a72b-114e6c7936ef" path="/var/lib/kubelet/pods/57703f44-9443-441c-a72b-114e6c7936ef/volumes"
Sep 06 06:13:30.787390 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.787374    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" path="/var/lib/kubelet/pods/5fdae1fd-80ff-4203-9ee8-3f5685de81ad/volumes"
Sep 06 06:13:30.801830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.801804    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"
Sep 06 06:13:30.801830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.801820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb" event={"ID":"76c2543c-0215-44b4-b359-64b772d3aa1d","Type":"ContainerDied","Data":"8629d23e78dafb4756c3b1f1f3243fb3f5b3ab0b69a7e013e494f9fd92cc4932"}
Sep 06 06:13:30.802044 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.801889    2567 scope.go:117] "RemoveContainer" containerID="4466ea3b08984cecba80a02356cc828b7515c6c0de0d175275cb7ec6ea18c3bb"
Sep 06 06:13:30.821024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.820993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:13:30.826607 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.826577    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f260fc83-7c6bd8dfd4-5cdgb"]
Sep 06 06:13:30.827657 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:30.827619    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:32.142546 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.142507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:32.347326 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.347281    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" podUID="d5df396d-aa84-46cc-a025-98cc2e35992b" containerName="main" containerID="cri-o://f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7" gracePeriod=45
Sep 06 06:13:32.600031 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.600006    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:13:32.683120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8tsf\" (UniqueName: \"kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683121    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683150    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683177    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683198    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683285    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache\") pod \"d5df396d-aa84-46cc-a025-98cc2e35992b\" (UID: \"d5df396d-aa84-46cc-a025-98cc2e35992b\") "
Sep 06 06:13:32.683573 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683419    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home" (OuterVolumeSpecName: "home") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:13:32.683573 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683432    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:13:32.683668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:13:32.683668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683598    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.683668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.683616    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.685497 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.685472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf" (OuterVolumeSpecName: "kube-api-access-w8tsf") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "kube-api-access-w8tsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:13:32.685608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.685508    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm" (OuterVolumeSpecName: "dshm") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:13:32.685608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.685547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5df396d-aa84-46cc-a025-98cc2e35992b" (UID: "d5df396d-aa84-46cc-a025-98cc2e35992b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:13:32.784445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.784350    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.784445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.784383    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8tsf\" (UniqueName: \"kubernetes.io/projected/d5df396d-aa84-46cc-a025-98cc2e35992b-kube-api-access-w8tsf\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.784445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.784399    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5df396d-aa84-46cc-a025-98cc2e35992b-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.784445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.784414    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5df396d-aa84-46cc-a025-98cc2e35992b-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:13:32.786739 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.786712    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76c2543c-0215-44b4-b359-64b772d3aa1d" path="/var/lib/kubelet/pods/76c2543c-0215-44b4-b359-64b772d3aa1d/volumes"
Sep 06 06:13:32.814982 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.814952    2567 generic.go:356] "Generic (PLEG): container finished" podID="d5df396d-aa84-46cc-a025-98cc2e35992b" containerID="f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7" exitCode=0
Sep 06 06:13:32.815203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.815005    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"
Sep 06 06:13:32.815203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.815033    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" event={"ID":"d5df396d-aa84-46cc-a025-98cc2e35992b","Type":"ContainerDied","Data":"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"}
Sep 06 06:13:32.815203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.815096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf" event={"ID":"d5df396d-aa84-46cc-a025-98cc2e35992b","Type":"ContainerDied","Data":"ed3c24780cb121e4b86efc0653dc80f887ee206ec78e7944d2d74fae2be74d19"}
Sep 06 06:13:32.815203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.815115    2567 scope.go:117] "RemoveContainer" containerID="f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"
Sep 06 06:13:32.824172 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.824153    2567 scope.go:117] "RemoveContainer" containerID="f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"
Sep 06 06:13:32.824512 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:13:32.824491    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7\": container with ID starting with f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7 not found: ID does not exist" containerID="f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"
Sep 06 06:13:32.824601 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.824521    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7"} err="failed to get container status \"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7\": rpc error: code = NotFound desc = could not find container \"f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7\": container with ID starting with f0c0ff83d70aa1a6215ff1dd7b74254fbb9325450100650a093934758507b4b7 not found: ID does not exist"
Sep 06 06:13:32.833852 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.833824    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:13:32.838203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:32.838177    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-e75768/autores-model-e75768-kserve-7f8d56f94c-nbpbf"]
Sep 06 06:13:34.787650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:34.787615    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5df396d-aa84-46cc-a025-98cc2e35992b" path="/var/lib/kubelet/pods/d5df396d-aa84-46cc-a025-98cc2e35992b/volumes"
Sep 06 06:13:34.815526 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:34.815493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:34.914523 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:34.914489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:35.044986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:35.044887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:35.471620 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:35.471582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:35.760938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:35.760844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:35.852189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:35.852158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.015334 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.015300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.075197 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.075148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.154307 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.154251    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.252966 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.252928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.353761 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.353720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.485680 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.485643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.581981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.581947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:38.774514 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:38.774429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:39.585410 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585378    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585722    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c2543c-0215-44b4-b359-64b772d3aa1d" containerName="maas-api"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585734    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c2543c-0215-44b4-b359-64b772d3aa1d" containerName="maas-api"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585748    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" containerName="payload-processing"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585753    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" containerName="payload-processing"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585760    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5df396d-aa84-46cc-a025-98cc2e35992b" containerName="main"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585772    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5df396d-aa84-46cc-a025-98cc2e35992b" containerName="main"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585783    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57703f44-9443-441c-a72b-114e6c7936ef" containerName="maas-api"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585789    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="57703f44-9443-441c-a72b-114e6c7936ef" containerName="maas-api"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585846    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fdae1fd-80ff-4203-9ee8-3f5685de81ad" containerName="payload-processing"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585854    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c2543c-0215-44b4-b359-64b772d3aa1d" containerName="maas-api"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585860    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5df396d-aa84-46cc-a025-98cc2e35992b" containerName="main"
Sep 06 06:13:39.585872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.585867    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="57703f44-9443-441c-a72b-114e6c7936ef" containerName="maas-api"
Sep 06 06:13:39.588903 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.588883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.591712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.591692    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-c50b6f-serving-cert\""
Sep 06 06:13:39.598285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.598221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:13:39.615524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.615486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:39.645680 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.645647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.645857 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.645689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.645857 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.645760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.645857 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.645847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k78h\" (UniqueName: \"kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.716465 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.716423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:39.746312 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.746280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.746501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.746326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.746501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.746369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.746501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.746414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k78h\" (UniqueName: \"kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.747002 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.746977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.748729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.748705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.748840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.748762    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.755423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.755397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k78h\" (UniqueName: \"kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h\") pod \"maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:39.815587 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.815538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:39.901689 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:39.901595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:40.126485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.126447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:40.182110 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.182074    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:13:40.182590 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:13:40.182563    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fad9724_a071_432e_8c06_88c3027a6732.slice/crio-25e0cafa2337975b5f82da8904c446767c00ab4d8e6359cebf0a844c79a15083 WatchSource:0}: Error finding container 25e0cafa2337975b5f82da8904c446767c00ab4d8e6359cebf0a844c79a15083: Status 404 returned error can't find the container with id 25e0cafa2337975b5f82da8904c446767c00ab4d8e6359cebf0a844c79a15083
Sep 06 06:13:40.191508 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.191476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:40.299141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.299106    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:13:40.303671 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.303647    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.310192 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.310162    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-c50b6f-dockercfg-zmbxg\""
Sep 06 06:13:40.310365 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.310161    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-c50b6f\""
Sep 06 06:13:40.315200 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.315176    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:13:40.351763 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.351704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.351962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.351812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-962rc\" (UniqueName: \"kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.453344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.453265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.453344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.453320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-962rc\" (UniqueName: \"kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.454001 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.453976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.461537 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.461512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-962rc\" (UniqueName: \"kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc\") pod \"payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.616658 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.616628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:40.862830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.862792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" event={"ID":"8fad9724-a071-432e-8c06-88c3027a6732","Type":"ContainerStarted","Data":"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"}
Sep 06 06:13:40.863028 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.862856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:40.863028 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.862874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" event={"ID":"8fad9724-a071-432e-8c06-88c3027a6732","Type":"ContainerStarted","Data":"25e0cafa2337975b5f82da8904c446767c00ab4d8e6359cebf0a844c79a15083"}
Sep 06 06:13:40.880412 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.880352    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" podStartSLOduration=1.475886827 podStartE2EDuration="1.880332736s" totalImagesPullingTime="404.445909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:13:40.879791137 +0000 UTC m=+1174.583343873" watchObservedRunningTime="2026-09-06 06:13:40.880332736 +0000 UTC m=+1174.583885475"
Sep 06 06:13:40.915391 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:40.915363    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:13:40.916886 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:13:40.916857    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01fbd012_c7fe_4ac3_8c21_0a9cb2b42ecc.slice/crio-bb33d39d159bb9e31e99072889acbfc1ab1612fd93eb280381180349900042c4 WatchSource:0}: Error finding container bb33d39d159bb9e31e99072889acbfc1ab1612fd93eb280381180349900042c4: Status 404 returned error can't find the container with id bb33d39d159bb9e31e99072889acbfc1ab1612fd93eb280381180349900042c4
Sep 06 06:13:41.871565 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:41.871528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" event={"ID":"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc","Type":"ContainerStarted","Data":"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"}
Sep 06 06:13:41.871994 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:41.871593    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" event={"ID":"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc","Type":"ContainerStarted","Data":"bb33d39d159bb9e31e99072889acbfc1ab1612fd93eb280381180349900042c4"}
Sep 06 06:13:41.871994 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:41.871770    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:41.889645 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:41.889600    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" podStartSLOduration=1.889587949 podStartE2EDuration="1.889587949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:13:41.888097621 +0000 UTC m=+1175.591650357" watchObservedRunningTime="2026-09-06 06:13:41.889587949 +0000 UTC m=+1175.593140686"
Sep 06 06:13:46.877484 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:46.877454    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:13:52.879592 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:52.879558    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:13:58.121548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:58.121508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:13:59.390385 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:13:59.390348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:03.381775 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:03.381733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:04.556692 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:04.556639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:10.133741 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:10.133699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:11.279564 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:11.279526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:23.506936 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.506879    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:23.512033 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.512013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.514616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.514597    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c50b6f\"/\"kube-root-ca.crt\""
Sep 06 06:14:23.514714 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.514683    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c50b6f\"/\"openshift-service-ca.crt\""
Sep 06 06:14:23.515924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.515884    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c50b6f\"/\"explicit-model-c50b6f-kserve-self-signed-certs\""
Sep 06 06:14:23.515924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.515917    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c50b6f\"/\"default-dockercfg-tszz5\""
Sep 06 06:14:23.519170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.519144    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:23.525241 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.525213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:23.543885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.543863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqmwn\" (UniqueName: \"kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.544024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.543903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.544024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.544012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.544184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.544090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.544184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.544129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.544293 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.544250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644788 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqmwn\" (UniqueName: \"kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.644922 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.644894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.645278 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.645257    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.645346 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.645312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.645403 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.645363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.647260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.647238    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.647449 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.647431    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.651961 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.651944    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqmwn\" (UniqueName: \"kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn\") pod \"explicit-model-c50b6f-kserve-67f878fdcc-7c4lr\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") " pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:23.824263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:23.824233    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:24.097160 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:24.097138    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:24.099035 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:14:24.099009    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aaa9cea_afc6_43a0_a00f_b7d969964022.slice/crio-3c5cf71006b85529cb757d7bc6d37443ca3ee411e3b0538ccd141d839d1ae541 WatchSource:0}: Error finding container 3c5cf71006b85529cb757d7bc6d37443ca3ee411e3b0538ccd141d839d1ae541: Status 404 returned error can't find the container with id 3c5cf71006b85529cb757d7bc6d37443ca3ee411e3b0538ccd141d839d1ae541
Sep 06 06:14:25.103264 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:25.103232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" event={"ID":"3aaa9cea-afc6-43a0-a00f-b7d969964022","Type":"ContainerStarted","Data":"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"}
Sep 06 06:14:25.103264 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:25.103274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" event={"ID":"3aaa9cea-afc6-43a0-a00f-b7d969964022","Type":"ContainerStarted","Data":"3c5cf71006b85529cb757d7bc6d37443ca3ee411e3b0538ccd141d839d1ae541"}
Sep 06 06:14:25.119965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:25.119917    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" podStartSLOduration=2.119902195 podStartE2EDuration="2.119902195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:14:25.118494987 +0000 UTC m=+1218.822047748" watchObservedRunningTime="2026-09-06 06:14:25.119902195 +0000 UTC m=+1218.823454930"
Sep 06 06:14:25.612512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:25.612467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:26.106319 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:26.106276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:27.193233 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:27.193196    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:27.209595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:27.209548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:32.057806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:32.057763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:32.162041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:32.161993    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:32.256823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:32.256783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:33.825152 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:33.825107    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:37.723404 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.723371    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:14:37.723882 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.723595    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" podUID="8fad9724-a071-432e-8c06-88c3027a6732" containerName="maas-api" containerID="cri-o://6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a" gracePeriod=30
Sep 06 06:14:37.750634 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.750607    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:37.789890 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.789851    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:14:37.790140 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.790114    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" podUID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" containerName="payload-pre-processing" containerID="cri-o://f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22" gracePeriod=30
Sep 06 06:14:37.813303 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.812991    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:37.913332 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.913280    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:37.971050 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:37.971006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:38.038795 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.038766    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:14:38.072504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.072477    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs\") pod \"8fad9724-a071-432e-8c06-88c3027a6732\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") "
Sep 06 06:14:38.072626 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.072534    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca\") pod \"8fad9724-a071-432e-8c06-88c3027a6732\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") "
Sep 06 06:14:38.072626 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.072570    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls\") pod \"8fad9724-a071-432e-8c06-88c3027a6732\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") "
Sep 06 06:14:38.072734 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.072685    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k78h\" (UniqueName: \"kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h\") pod \"8fad9724-a071-432e-8c06-88c3027a6732\" (UID: \"8fad9724-a071-432e-8c06-88c3027a6732\") "
Sep 06 06:14:38.073427 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.073401    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8fad9724-a071-432e-8c06-88c3027a6732" (UID: "8fad9724-a071-432e-8c06-88c3027a6732"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:14:38.075282 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.075253    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h" (OuterVolumeSpecName: "kube-api-access-5k78h") pod "8fad9724-a071-432e-8c06-88c3027a6732" (UID: "8fad9724-a071-432e-8c06-88c3027a6732"). InnerVolumeSpecName "kube-api-access-5k78h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:14:38.075473 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.075448    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8fad9724-a071-432e-8c06-88c3027a6732" (UID: "8fad9724-a071-432e-8c06-88c3027a6732"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:14:38.076006 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.075979    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8fad9724-a071-432e-8c06-88c3027a6732" (UID: "8fad9724-a071-432e-8c06-88c3027a6732"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:14:38.083409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.083390    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:14:38.173296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173268    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume\") pod \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") "
Sep 06 06:14:38.173495 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173352    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-962rc\" (UniqueName: \"kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc\") pod \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\" (UID: \"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc\") "
Sep 06 06:14:38.173608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173584    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k78h\" (UniqueName: \"kubernetes.io/projected/8fad9724-a071-432e-8c06-88c3027a6732-kube-api-access-5k78h\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.173608 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173610    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.173817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173624    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fad9724-a071-432e-8c06-88c3027a6732-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.173817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173633    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fad9724-a071-432e-8c06-88c3027a6732-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.173817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.173725    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" (UID: "01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:14:38.174596 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.174574    2567 generic.go:356] "Generic (PLEG): container finished" podID="8fad9724-a071-432e-8c06-88c3027a6732" containerID="6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a" exitCode=0
Sep 06 06:14:38.174697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.174618    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"
Sep 06 06:14:38.174697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.174642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" event={"ID":"8fad9724-a071-432e-8c06-88c3027a6732","Type":"ContainerDied","Data":"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"}
Sep 06 06:14:38.174817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.174717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj" event={"ID":"8fad9724-a071-432e-8c06-88c3027a6732","Type":"ContainerDied","Data":"25e0cafa2337975b5f82da8904c446767c00ab4d8e6359cebf0a844c79a15083"}
Sep 06 06:14:38.174817 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.174746    2567 scope.go:117] "RemoveContainer" containerID="6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"
Sep 06 06:14:38.175980 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.175869    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc" (OuterVolumeSpecName: "kube-api-access-962rc") pod "01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" (UID: "01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc"). InnerVolumeSpecName "kube-api-access-962rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:14:38.176144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.176122    2567 generic.go:356] "Generic (PLEG): container finished" podID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" containerID="f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22" exitCode=0
Sep 06 06:14:38.176218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.176170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" event={"ID":"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc","Type":"ContainerDied","Data":"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"}
Sep 06 06:14:38.176218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.176184    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"
Sep 06 06:14:38.176218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.176216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4" event={"ID":"01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc","Type":"ContainerDied","Data":"bb33d39d159bb9e31e99072889acbfc1ab1612fd93eb280381180349900042c4"}
Sep 06 06:14:38.185105 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.185083    2567 scope.go:117] "RemoveContainer" containerID="6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"
Sep 06 06:14:38.185447 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:14:38.185424    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a\": container with ID starting with 6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a not found: ID does not exist" containerID="6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"
Sep 06 06:14:38.185537 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.185454    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a"} err="failed to get container status \"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a\": rpc error: code = NotFound desc = could not find container \"6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a\": container with ID starting with 6cbbe215791cfa3a077fb429b50f210027b16081a5dcf67274960923c330621a not found: ID does not exist"
Sep 06 06:14:38.185537 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.185472    2567 scope.go:117] "RemoveContainer" containerID="f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"
Sep 06 06:14:38.193996 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.193975    2567 scope.go:117] "RemoveContainer" containerID="f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"
Sep 06 06:14:38.194333 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:14:38.194310    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22\": container with ID starting with f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22 not found: ID does not exist" containerID="f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"
Sep 06 06:14:38.194409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.194343    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22"} err="failed to get container status \"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22\": rpc error: code = NotFound desc = could not find container \"f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22\": container with ID starting with f49ec26fdfc7b0a38d6cdf8f191bba3dcb113513871b626a99c97aad749c2a22 not found: ID does not exist"
Sep 06 06:14:38.202294 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.202266    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:14:38.208260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.208234    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c50b6f-5966bf55d9-ll6pj"]
Sep 06 06:14:38.217860 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.217831    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:14:38.221663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.221636    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c50b6f-68bfd47674-m78t4"]
Sep 06 06:14:38.274784 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.274697    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-962rc\" (UniqueName: \"kubernetes.io/projected/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-kube-api-access-962rc\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.274784 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.274726    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:38.281235 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.281201    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:38.786590 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.786553    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" path="/var/lib/kubelet/pods/01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc/volumes"
Sep 06 06:14:38.787188 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:38.787166    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fad9724-a071-432e-8c06-88c3027a6732" path="/var/lib/kubelet/pods/8fad9724-a071-432e-8c06-88c3027a6732/volumes"
Sep 06 06:14:42.214110 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.214049    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" podUID="3aaa9cea-afc6-43a0-a00f-b7d969964022" containerName="main" containerID="cri-o://7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806" gracePeriod=45
Sep 06 06:14:42.467363 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.467300    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:42.513408 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513374    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513427    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513484    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513509    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513537    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqmwn\" (UniqueName: \"kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn\") pod \"3aaa9cea-afc6-43a0-a00f-b7d969964022\" (UID: \"3aaa9cea-afc6-43a0-a00f-b7d969964022\") "
Sep 06 06:14:42.513887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513754    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home" (OuterVolumeSpecName: "home") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:14:42.513887 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513779    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache" (OuterVolumeSpecName: "model-cache") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:14:42.513989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.513912    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:14:42.515750 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.515726    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:14:42.515750 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.515734    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm" (OuterVolumeSpecName: "dshm") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:14:42.515897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.515752    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn" (OuterVolumeSpecName: "kube-api-access-xqmwn") pod "3aaa9cea-afc6-43a0-a00f-b7d969964022" (UID: "3aaa9cea-afc6-43a0-a00f-b7d969964022"). InnerVolumeSpecName "kube-api-access-xqmwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:14:42.614557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614520    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.614557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614551    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqmwn\" (UniqueName: \"kubernetes.io/projected/3aaa9cea-afc6-43a0-a00f-b7d969964022-kube-api-access-xqmwn\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.614557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614563    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaa9cea-afc6-43a0-a00f-b7d969964022-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.614797 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614571    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.614797 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614579    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.614797 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.614586    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaa9cea-afc6-43a0-a00f-b7d969964022-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:14:42.707369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:42.707332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:43.206430 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.206389    2567 generic.go:356] "Generic (PLEG): container finished" podID="3aaa9cea-afc6-43a0-a00f-b7d969964022" containerID="7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806" exitCode=0
Sep 06 06:14:43.206430 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.206435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" event={"ID":"3aaa9cea-afc6-43a0-a00f-b7d969964022","Type":"ContainerDied","Data":"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"}
Sep 06 06:14:43.206668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.206444    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"
Sep 06 06:14:43.206668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.206476    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr" event={"ID":"3aaa9cea-afc6-43a0-a00f-b7d969964022","Type":"ContainerDied","Data":"3c5cf71006b85529cb757d7bc6d37443ca3ee411e3b0538ccd141d839d1ae541"}
Sep 06 06:14:43.206668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.206496    2567 scope.go:117] "RemoveContainer" containerID="7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"
Sep 06 06:14:43.215579 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.215396    2567 scope.go:117] "RemoveContainer" containerID="7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"
Sep 06 06:14:43.215871 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:14:43.215719    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806\": container with ID starting with 7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806 not found: ID does not exist" containerID="7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"
Sep 06 06:14:43.215871 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.215747    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806"} err="failed to get container status \"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806\": rpc error: code = NotFound desc = could not find container \"7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806\": container with ID starting with 7299d050bbadbcd2e895abfaceb370e3bcd6fcc404d6a699c6e6c4d8766d4806 not found: ID does not exist"
Sep 06 06:14:43.226361 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.226321    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:43.229077 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.229034    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-c50b6f/explicit-model-c50b6f-kserve-67f878fdcc-7c4lr"]
Sep 06 06:14:43.447989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.447958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:43.897344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.897309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:43.960711 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:43.960676    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:44.133891 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:44.133856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:44.588923 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:44.588890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:44.788185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:44.788150    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aaa9cea-afc6-43a0-a00f-b7d969964022" path="/var/lib/kubelet/pods/3aaa9cea-afc6-43a0-a00f-b7d969964022/volumes"
Sep 06 06:14:44.891452 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:44.891365    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:45.259350 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:45.259266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:46.312493 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:46.312462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:46.413217 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:46.413185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:46.523863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:46.523819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:49.232737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:49.232705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:49.281731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:49.281697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:49.364488 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:49.364456    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:49.464103 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:49.464031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:49.564468 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:49.564437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:50.169716 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:50.169659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:50.267555 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:50.267515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:50.368416 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:50.368380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:51.770007 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:51.769967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:57.370206 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:57.370171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:57.429258 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:57.429222    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:14:58.031619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:14:58.031584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:03.773389 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:03.773304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:03.835781 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:03.835751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:04.003892 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:04.003852    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:04.292208 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:04.292170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:04.739250 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:04.739209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:07.053826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:07.053775    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:07.148962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:07.148870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:08.173108 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:08.173524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173472    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aaa9cea-afc6-43a0-a00f-b7d969964022" containerName="main"
Sep 06 06:15:08.173524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173491    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aaa9cea-afc6-43a0-a00f-b7d969964022" containerName="main"
Sep 06 06:15:08.173524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173505    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fad9724-a071-432e-8c06-88c3027a6732" containerName="maas-api"
Sep 06 06:15:08.173524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173512    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fad9724-a071-432e-8c06-88c3027a6732" containerName="maas-api"
Sep 06 06:15:08.173737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173521    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" containerName="payload-pre-processing"
Sep 06 06:15:08.173737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173561    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" containerName="payload-pre-processing"
Sep 06 06:15:08.173737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173663    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aaa9cea-afc6-43a0-a00f-b7d969964022" containerName="main"
Sep 06 06:15:08.173737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173680    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fad9724-a071-432e-8c06-88c3027a6732" containerName="maas-api"
Sep 06 06:15:08.173737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.173688    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01fbd012-c7fe-4ac3-8c21-0a9cb2b42ecc" containerName="payload-pre-processing"
Sep 06 06:15:08.178157 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.178134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.180987 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.180968    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fc95779d-serving-cert\""
Sep 06 06:15:08.187872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.187847    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:08.214324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.214292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:08.312329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.312297    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:08.346244 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.346211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.346244 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.346256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58rfh\" (UniqueName: \"kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.346443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.346314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.346443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.346419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.447668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.447580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.447668 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.447628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.447862 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:08.447730    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-serving-cert: secret "maas-api-e2e-mt-fc95779d-serving-cert" not found
Sep 06 06:15:08.447862 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:08.447805    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls podName:bbda4532-b0df-4a1b-b561-a3a5167a26b5 nodeName:}" failed. No retries permitted until 2026-09-06 06:15:08.94778236 +0000 UTC m=+1262.651335074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls") pod "maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5") : secret "maas-api-e2e-mt-fc95779d-serving-cert" not found
Sep 06 06:15:08.447938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.447882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58rfh\" (UniqueName: \"kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.448052 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.448027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.448509 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.448493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.450512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.450483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.458081 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.458046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58rfh\" (UniqueName: \"kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.669456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.669424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:08.772503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.772419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:08.786738 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.786711    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:08.790242 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.790221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:08.792963 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.792938    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fc95779d-dockercfg-66dx6\""
Sep 06 06:15:08.793128 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.793097    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fc95779d\""
Sep 06 06:15:08.796200 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.796176    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:08.952848 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.952818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:08.953017 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.952859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qdv2\" (UniqueName: \"kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:08.953017 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:08.952946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:08.953139 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:08.953038    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-serving-cert: secret "maas-api-e2e-mt-fc95779d-serving-cert" not found
Sep 06 06:15:08.953139 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:08.953129    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls podName:bbda4532-b0df-4a1b-b561-a3a5167a26b5 nodeName:}" failed. No retries permitted until 2026-09-06 06:15:09.953112115 +0000 UTC m=+1263.656664829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls") pod "maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5") : secret "maas-api-e2e-mt-fc95779d-serving-cert" not found
Sep 06 06:15:09.053505 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.053402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:09.053505 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.053461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qdv2\" (UniqueName: \"kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:09.054252 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.054232    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:09.062032 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.062008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qdv2\" (UniqueName: \"kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2\") pod \"payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:09.070221 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.070194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:09.102207 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.102170    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:09.374771 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.374739    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:09.376113 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:15:09.376084    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539704ca_b437_4147_ace2_a7536983e684.slice/crio-a5796733ed0c7fc8374d7a40da17514cb851308ed1540bb809622f74510f95f1 WatchSource:0}: Error finding container a5796733ed0c7fc8374d7a40da17514cb851308ed1540bb809622f74510f95f1: Status 404 returned error can't find the container with id a5796733ed0c7fc8374d7a40da17514cb851308ed1540bb809622f74510f95f1
Sep 06 06:15:09.962572 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.962538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:09.964975 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.964957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:09.993050 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:09.993014    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:10.266371 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.266339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:10.267609 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:15:10.267580    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbda4532_b0df_4a1b_b561_a3a5167a26b5.slice/crio-6ed35ed6e3e55f399322fda2f904358e717160387d065594e99b704fc53ab445 WatchSource:0}: Error finding container 6ed35ed6e3e55f399322fda2f904358e717160387d065594e99b704fc53ab445: Status 404 returned error can't find the container with id 6ed35ed6e3e55f399322fda2f904358e717160387d065594e99b704fc53ab445
Sep 06 06:15:10.359139 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.359096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" event={"ID":"bbda4532-b0df-4a1b-b561-a3a5167a26b5","Type":"ContainerStarted","Data":"6ed35ed6e3e55f399322fda2f904358e717160387d065594e99b704fc53ab445"}
Sep 06 06:15:10.360702 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.360672    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" event={"ID":"539704ca-b437-4147-ace2-a7536983e684","Type":"ContainerStarted","Data":"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"}
Sep 06 06:15:10.360819 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.360715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" event={"ID":"539704ca-b437-4147-ace2-a7536983e684","Type":"ContainerStarted","Data":"a5796733ed0c7fc8374d7a40da17514cb851308ed1540bb809622f74510f95f1"}
Sep 06 06:15:10.360870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.360818    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:10.379175 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:10.379130    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" podStartSLOduration=2.379111547 podStartE2EDuration="2.379111547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:15:10.376723125 +0000 UTC m=+1264.080275861" watchObservedRunningTime="2026-09-06 06:15:10.379111547 +0000 UTC m=+1264.082664284"
Sep 06 06:15:11.367249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.367213    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" event={"ID":"bbda4532-b0df-4a1b-b561-a3a5167a26b5","Type":"ContainerStarted","Data":"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"}
Sep 06 06:15:11.367439 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.367316    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:11.383818 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.383770    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" podStartSLOduration=2.987607755 podStartE2EDuration="3.38375215s" totalImagesPullingTime="396.144395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:15:11.38327248 +0000 UTC m=+1265.086825209" watchObservedRunningTime="2026-09-06 06:15:11.38375215 +0000 UTC m=+1265.087304890"
Sep 06 06:15:11.490594 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.490537    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:11.494452 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.494429    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.497647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.497542    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-837ae9\"/\"default-dockercfg-v67mf\""
Sep 06 06:15:11.497647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.497604    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-837ae9\"/\"kube-root-ca.crt\""
Sep 06 06:15:11.497967 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.497656    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-837ae9\"/\"noten-model-837ae9-kserve-self-signed-certs\""
Sep 06 06:15:11.497967 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.497659    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-837ae9\"/\"openshift-service-ca.crt\""
Sep 06 06:15:11.503495 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.503473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:11.575722 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575672    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.575722 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.575950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfhcm\" (UniqueName: \"kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.575950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.575950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.575950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.575883    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.676860 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.676762    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.676860 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.676808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677109 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.676937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfhcm\" (UniqueName: \"kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677109 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.676995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677109 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.677072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.677115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677404 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.677384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677459 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.677419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.677622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.677596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.679299 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.679272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.679491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.679474    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.684902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.684879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfhcm\" (UniqueName: \"kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm\") pod \"noten-model-837ae9-kserve-67b6f75947-9tbz4\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") " pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:11.696165 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.696136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:11.806165 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:11.806131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:12.080018 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:12.079856    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:12.082586 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:15:12.082551    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64b840e5_5d35_4271_8a30_c3c2107fcf3f.slice/crio-850e6abcb33806d667f00298400017dad9b0443a840a78f6440955bdfdfa0bfa WatchSource:0}: Error finding container 850e6abcb33806d667f00298400017dad9b0443a840a78f6440955bdfdfa0bfa: Status 404 returned error can't find the container with id 850e6abcb33806d667f00298400017dad9b0443a840a78f6440955bdfdfa0bfa
Sep 06 06:15:12.375203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:12.375106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" event={"ID":"64b840e5-5d35-4271-8a30-c3c2107fcf3f","Type":"ContainerStarted","Data":"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"}
Sep 06 06:15:12.375203 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:12.375156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" event={"ID":"64b840e5-5d35-4271-8a30-c3c2107fcf3f","Type":"ContainerStarted","Data":"850e6abcb33806d667f00298400017dad9b0443a840a78f6440955bdfdfa0bfa"}
Sep 06 06:15:12.392000 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:12.391944    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" podStartSLOduration=1.3919285719999999 podStartE2EDuration="1.391928572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:15:12.390261155 +0000 UTC m=+1266.093813916" watchObservedRunningTime="2026-09-06 06:15:12.391928572 +0000 UTC m=+1266.095481308"
Sep 06 06:15:15.411135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:15.411094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:16.603071 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:16.603023    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:16.615380 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:16.615353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:16.734650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:16.734611    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:16.948184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:16.948085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:17.056046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:17.056011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:17.381361 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:17.381335    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:17.803667 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:17.803585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:18.015593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:18.015557    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:21.369419 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:21.369391    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:21.807005 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:21.806982    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:22.132053 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:22.131954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.322836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.322800    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:15:28.323461 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.323015    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" podUID="55b2c4d6-632d-4137-886f-970d10618a66" containerName="maas-api" containerID="cri-o://57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0" gracePeriod=30
Sep 06 06:15:28.353935 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.353900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.381448 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.381416    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:15:28.381712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.381680    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" podUID="5b7ca98f-151f-4e34-8b28-b37e2318126b" containerName="payload-processing" containerID="cri-o://3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7" gracePeriod=30
Sep 06 06:15:28.388680 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.388648    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:15:28.388897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.388875    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" podUID="8e6567d6-997b-4d57-84b4-74e2126266a5" containerName="payload-pre-processing" containerID="cri-o://38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c" gracePeriod=30
Sep 06 06:15:28.422220 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.422186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.481096 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.481045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.544547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.544475    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.599372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.599286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.672520 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.672498    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:15:28.686215 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.686196    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:15:28.737087 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737026    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8llpn\" (UniqueName: \"kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn\") pod \"8e6567d6-997b-4d57-84b4-74e2126266a5\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") "
Sep 06 06:15:28.737234 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737114    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wtnb\" (UniqueName: \"kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb\") pod \"5b7ca98f-151f-4e34-8b28-b37e2318126b\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") "
Sep 06 06:15:28.737234 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737137    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume\") pod \"8e6567d6-997b-4d57-84b4-74e2126266a5\" (UID: \"8e6567d6-997b-4d57-84b4-74e2126266a5\") "
Sep 06 06:15:28.737234 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737167    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume\") pod \"5b7ca98f-151f-4e34-8b28-b37e2318126b\" (UID: \"5b7ca98f-151f-4e34-8b28-b37e2318126b\") "
Sep 06 06:15:28.737754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737729    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b7ca98f-151f-4e34-8b28-b37e2318126b" (UID: "5b7ca98f-151f-4e34-8b28-b37e2318126b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:28.737865 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.737743    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e6567d6-997b-4d57-84b4-74e2126266a5" (UID: "8e6567d6-997b-4d57-84b4-74e2126266a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:28.739619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.739582    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb" (OuterVolumeSpecName: "kube-api-access-5wtnb") pod "5b7ca98f-151f-4e34-8b28-b37e2318126b" (UID: "5b7ca98f-151f-4e34-8b28-b37e2318126b"). InnerVolumeSpecName "kube-api-access-5wtnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:28.739619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.739617    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn" (OuterVolumeSpecName: "kube-api-access-8llpn") pod "8e6567d6-997b-4d57-84b4-74e2126266a5" (UID: "8e6567d6-997b-4d57-84b4-74e2126266a5"). InnerVolumeSpecName "kube-api-access-8llpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:28.803186 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.803160    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:15:28.838510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.838471    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7ca98f-151f-4e34-8b28-b37e2318126b-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:28.838510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.838502    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8llpn\" (UniqueName: \"kubernetes.io/projected/8e6567d6-997b-4d57-84b4-74e2126266a5-kube-api-access-8llpn\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:28.838510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.838516    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wtnb\" (UniqueName: \"kubernetes.io/projected/5b7ca98f-151f-4e34-8b28-b37e2318126b-kube-api-access-5wtnb\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:28.838760 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.838529    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e6567d6-997b-4d57-84b4-74e2126266a5-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:28.873547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.873457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:28.939712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.939669    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls\") pod \"55b2c4d6-632d-4137-886f-970d10618a66\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") "
Sep 06 06:15:28.939913 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.939803    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca\") pod \"55b2c4d6-632d-4137-886f-970d10618a66\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") "
Sep 06 06:15:28.939913 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.939836    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs\") pod \"55b2c4d6-632d-4137-886f-970d10618a66\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") "
Sep 06 06:15:28.939913 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.939862    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69slk\" (UniqueName: \"kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk\") pod \"55b2c4d6-632d-4137-886f-970d10618a66\" (UID: \"55b2c4d6-632d-4137-886f-970d10618a66\") "
Sep 06 06:15:28.940274 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.940243    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55b2c4d6-632d-4137-886f-970d10618a66" (UID: "55b2c4d6-632d-4137-886f-970d10618a66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:28.941990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.941954    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk" (OuterVolumeSpecName: "kube-api-access-69slk") pod "55b2c4d6-632d-4137-886f-970d10618a66" (UID: "55b2c4d6-632d-4137-886f-970d10618a66"). InnerVolumeSpecName "kube-api-access-69slk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:28.941990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.941969    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55b2c4d6-632d-4137-886f-970d10618a66" (UID: "55b2c4d6-632d-4137-886f-970d10618a66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:28.942304 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:28.942289    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55b2c4d6-632d-4137-886f-970d10618a66" (UID: "55b2c4d6-632d-4137-886f-970d10618a66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:29.041548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.041513    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:29.041548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.041542    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b2c4d6-632d-4137-886f-970d10618a66-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:29.041548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.041551    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b2c4d6-632d-4137-886f-970d10618a66-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:29.041548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.041560    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69slk\" (UniqueName: \"kubernetes.io/projected/55b2c4d6-632d-4137-886f-970d10618a66-kube-api-access-69slk\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:29.051560 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.051524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:29.470415 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.470380    2567 generic.go:356] "Generic (PLEG): container finished" podID="55b2c4d6-632d-4137-886f-970d10618a66" containerID="57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0" exitCode=0
Sep 06 06:15:29.470840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.470427    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"
Sep 06 06:15:29.470840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.470465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" event={"ID":"55b2c4d6-632d-4137-886f-970d10618a66","Type":"ContainerDied","Data":"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"}
Sep 06 06:15:29.470840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.470518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql" event={"ID":"55b2c4d6-632d-4137-886f-970d10618a66","Type":"ContainerDied","Data":"3655f80e8c10d4c9bbc23e563576a0ddf3d1bb44ebeba92c2d87eacff410a684"}
Sep 06 06:15:29.470840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.470546    2567 scope.go:117] "RemoveContainer" containerID="57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"
Sep 06 06:15:29.472040 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.472020    2567 generic.go:356] "Generic (PLEG): container finished" podID="5b7ca98f-151f-4e34-8b28-b37e2318126b" containerID="3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7" exitCode=0
Sep 06 06:15:29.472152 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.472086    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"
Sep 06 06:15:29.472152 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.472108    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" event={"ID":"5b7ca98f-151f-4e34-8b28-b37e2318126b","Type":"ContainerDied","Data":"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"}
Sep 06 06:15:29.472273 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.472154    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv" event={"ID":"5b7ca98f-151f-4e34-8b28-b37e2318126b","Type":"ContainerDied","Data":"68c39d9b725f9d5acd435b1eea869d13cf25474a8467384f527b99f968beca9b"}
Sep 06 06:15:29.473806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.473783    2567 generic.go:356] "Generic (PLEG): container finished" podID="8e6567d6-997b-4d57-84b4-74e2126266a5" containerID="38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c" exitCode=0
Sep 06 06:15:29.473988 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.473816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" event={"ID":"8e6567d6-997b-4d57-84b4-74e2126266a5","Type":"ContainerDied","Data":"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"}
Sep 06 06:15:29.474159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.474141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls" event={"ID":"8e6567d6-997b-4d57-84b4-74e2126266a5","Type":"ContainerDied","Data":"990eff5b83f324f8b905bef1496adf5d961cec6161d468cedbcd4a7c8fe880e0"}
Sep 06 06:15:29.474302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.473831    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"
Sep 06 06:15:29.481995 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.481979    2567 scope.go:117] "RemoveContainer" containerID="57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"
Sep 06 06:15:29.482303 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:29.482278    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0\": container with ID starting with 57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0 not found: ID does not exist" containerID="57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"
Sep 06 06:15:29.482376 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.482306    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0"} err="failed to get container status \"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0\": rpc error: code = NotFound desc = could not find container \"57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0\": container with ID starting with 57d5000d97faa97355a742e3c68759b2a471aec0bb65de73552e7799362dd5d0 not found: ID does not exist"
Sep 06 06:15:29.482376 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.482330    2567 scope.go:117] "RemoveContainer" containerID="3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"
Sep 06 06:15:29.490357 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.490328    2567 scope.go:117] "RemoveContainer" containerID="3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"
Sep 06 06:15:29.490855 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:29.490814    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7\": container with ID starting with 3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7 not found: ID does not exist" containerID="3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"
Sep 06 06:15:29.490977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.490864    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7"} err="failed to get container status \"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7\": rpc error: code = NotFound desc = could not find container \"3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7\": container with ID starting with 3477fa0d83bbc955c27ae031e24827df22228fac1b45e662315b87d88c6bb6c7 not found: ID does not exist"
Sep 06 06:15:29.490977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.490887    2567 scope.go:117] "RemoveContainer" containerID="38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"
Sep 06 06:15:29.492967 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.492947    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:15:29.499254 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.499234    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ceca3b-6fffb84947-58nxv"]
Sep 06 06:15:29.499663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.499640    2567 scope.go:117] "RemoveContainer" containerID="38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"
Sep 06 06:15:29.500036 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:29.500016    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c\": container with ID starting with 38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c not found: ID does not exist" containerID="38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"
Sep 06 06:15:29.500151 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.500042    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c"} err="failed to get container status \"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c\": rpc error: code = NotFound desc = could not find container \"38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c\": container with ID starting with 38f9f70b7049f1cc5ee919c29b7817dba7d022eb9c7a05c2fd5a61779efef45c not found: ID does not exist"
Sep 06 06:15:29.512807 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.512781    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:15:29.520379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.520348    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ceca3b-84fddf5f65-swvql"]
Sep 06 06:15:29.526416 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.526391    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:15:29.534165 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:29.534141    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ceca3b-65965b7654-8vrls"]
Sep 06 06:15:30.787231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:30.787199    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b2c4d6-632d-4137-886f-970d10618a66" path="/var/lib/kubelet/pods/55b2c4d6-632d-4137-886f-970d10618a66/volumes"
Sep 06 06:15:30.787606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:30.787593    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7ca98f-151f-4e34-8b28-b37e2318126b" path="/var/lib/kubelet/pods/5b7ca98f-151f-4e34-8b28-b37e2318126b/volumes"
Sep 06 06:15:30.787941 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:30.787926    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e6567d6-997b-4d57-84b4-74e2126266a5" path="/var/lib/kubelet/pods/8e6567d6-997b-4d57-84b4-74e2126266a5/volumes"
Sep 06 06:15:31.623727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.623685    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" podUID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" containerName="main" containerID="cri-o://6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573" gracePeriod=45
Sep 06 06:15:31.870647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.870617    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:31.967552 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967474    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfhcm\" (UniqueName: \"kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967552 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967513    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967585    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967615    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967663    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967740    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache\") pod \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\" (UID: \"64b840e5-5d35-4271-8a30-c3c2107fcf3f\") "
Sep 06 06:15:31.967885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967834    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home" (OuterVolumeSpecName: "home") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:15:31.967885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:15:31.968049 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.967990    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:31.968049 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.968005    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:31.968049 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.968036    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:15:31.969978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.969950    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:31.970114 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.969991    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm" (OuterVolumeSpecName: "dshm") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:15:31.970114 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.969998    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm" (OuterVolumeSpecName: "kube-api-access-bfhcm") pod "64b840e5-5d35-4271-8a30-c3c2107fcf3f" (UID: "64b840e5-5d35-4271-8a30-c3c2107fcf3f"). InnerVolumeSpecName "kube-api-access-bfhcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:31.994441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:31.994405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:32.069464 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.069423    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64b840e5-5d35-4271-8a30-c3c2107fcf3f-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:32.069464 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.069456    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:32.069464 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.069470    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfhcm\" (UniqueName: \"kubernetes.io/projected/64b840e5-5d35-4271-8a30-c3c2107fcf3f-kube-api-access-bfhcm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:32.069699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.069482    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64b840e5-5d35-4271-8a30-c3c2107fcf3f-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:32.494547 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.494517    2567 generic.go:356] "Generic (PLEG): container finished" podID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" containerID="6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573" exitCode=0
Sep 06 06:15:32.494723 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.494558    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"
Sep 06 06:15:32.494723 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.494582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" event={"ID":"64b840e5-5d35-4271-8a30-c3c2107fcf3f","Type":"ContainerDied","Data":"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"}
Sep 06 06:15:32.494723 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.494616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4" event={"ID":"64b840e5-5d35-4271-8a30-c3c2107fcf3f","Type":"ContainerDied","Data":"850e6abcb33806d667f00298400017dad9b0443a840a78f6440955bdfdfa0bfa"}
Sep 06 06:15:32.494723 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.494634    2567 scope.go:117] "RemoveContainer" containerID="6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"
Sep 06 06:15:32.504122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.504104    2567 scope.go:117] "RemoveContainer" containerID="6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"
Sep 06 06:15:32.504377 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:32.504361    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573\": container with ID starting with 6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573 not found: ID does not exist" containerID="6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"
Sep 06 06:15:32.504431 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.504384    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573"} err="failed to get container status \"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573\": rpc error: code = NotFound desc = could not find container \"6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573\": container with ID starting with 6f41eacc1d0159ede87f2e814a7af0c62a682154f0214a5bc570a0344af6d573 not found: ID does not exist"
Sep 06 06:15:32.517325 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.517293    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:32.520411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.520390    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-837ae9/noten-model-837ae9-kserve-67b6f75947-9tbz4"]
Sep 06 06:15:32.622519 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.622483    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:32.721933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.721896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:32.786442 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:32.786355    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" path="/var/lib/kubelet/pods/64b840e5-5d35-4271-8a30-c3c2107fcf3f/volumes"
Sep 06 06:15:33.088302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:33.088272    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:34.759119 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:34.759089    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:34.815690 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:34.815656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:35.019807 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:35.019715    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:35.495669 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:35.495636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:38.836576 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:38.836539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:38.935581 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:38.935545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:39.296179 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:39.296080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:45.474712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:45.474669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.053362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.053328    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:15:46.053575 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.053552    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" containerName="maas-api" containerID="cri-o://f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8" gracePeriod=30
Sep 06 06:15:46.081477 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.081442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.106872 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.106823    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.47:8443/health\": dial tcp 10.134.0.47:8443: connect: connection refused"
Sep 06 06:15:46.148897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.148819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.209648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.209613    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.270544 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.270510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.301515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.301494    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:15:46.327021 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.326987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.398019 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.397988    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca\") pod \"89b42528-4b57-4e46-88bc-81834e8652ce\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") "
Sep 06 06:15:46.398019 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.398022    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsfmg\" (UniqueName: \"kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg\") pod \"89b42528-4b57-4e46-88bc-81834e8652ce\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") "
Sep 06 06:15:46.398306 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.398044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") pod \"89b42528-4b57-4e46-88bc-81834e8652ce\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") "
Sep 06 06:15:46.398306 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.398179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs\") pod \"89b42528-4b57-4e46-88bc-81834e8652ce\" (UID: \"89b42528-4b57-4e46-88bc-81834e8652ce\") "
Sep 06 06:15:46.398508 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.398485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89b42528-4b57-4e46-88bc-81834e8652ce" (UID: "89b42528-4b57-4e46-88bc-81834e8652ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:46.400305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.400278    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89b42528-4b57-4e46-88bc-81834e8652ce" (UID: "89b42528-4b57-4e46-88bc-81834e8652ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:46.400405 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.400310    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "89b42528-4b57-4e46-88bc-81834e8652ce" (UID: "89b42528-4b57-4e46-88bc-81834e8652ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:46.400405 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.400330    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg" (OuterVolumeSpecName: "kube-api-access-jsfmg") pod "89b42528-4b57-4e46-88bc-81834e8652ce" (UID: "89b42528-4b57-4e46-88bc-81834e8652ce"). InnerVolumeSpecName "kube-api-access-jsfmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:46.499700 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.499663    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:46.499700 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.499695    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b42528-4b57-4e46-88bc-81834e8652ce-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:46.499700 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.499705    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsfmg\" (UniqueName: \"kubernetes.io/projected/89b42528-4b57-4e46-88bc-81834e8652ce-kube-api-access-jsfmg\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:46.500196 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.499714    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b42528-4b57-4e46-88bc-81834e8652ce-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:46.579332 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.579246    2567 generic.go:356] "Generic (PLEG): container finished" podID="89b42528-4b57-4e46-88bc-81834e8652ce" containerID="f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8" exitCode=0
Sep 06 06:15:46.579332 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.579289    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"
Sep 06 06:15:46.579524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.579339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" event={"ID":"89b42528-4b57-4e46-88bc-81834e8652ce","Type":"ContainerDied","Data":"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"}
Sep 06 06:15:46.579524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.579394    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc" event={"ID":"89b42528-4b57-4e46-88bc-81834e8652ce","Type":"ContainerDied","Data":"d42eb1fccbe9e49a27d22435e62b6702a9ee3b97cdee8a71bd73d0e2c6064df8"}
Sep 06 06:15:46.579524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.579422    2567 scope.go:117] "RemoveContainer" containerID="f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"
Sep 06 06:15:46.588879 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.588862    2567 scope.go:117] "RemoveContainer" containerID="f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"
Sep 06 06:15:46.589161 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:46.589143    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8\": container with ID starting with f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8 not found: ID does not exist" containerID="f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"
Sep 06 06:15:46.589213 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.589169    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8"} err="failed to get container status \"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8\": rpc error: code = NotFound desc = could not find container \"f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8\": container with ID starting with f54c2ccef715a298e0073f181bba5b650a3b7973a6a00e662609ed7a720bccc8 not found: ID does not exist"
Sep 06 06:15:46.600443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.600416    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:15:46.603719 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.603698    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0bce0b-7b98d6976c-9cqcc"]
Sep 06 06:15:46.615989 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.615954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:46.787184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:46.787149    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" path="/var/lib/kubelet/pods/89b42528-4b57-4e46-88bc-81834e8652ce/volumes"
Sep 06 06:15:49.402586 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.402529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:49.923694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.923661    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:49.923920 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.923897    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" podUID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" containerName="maas-api" containerID="cri-o://2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75" gracePeriod=30
Sep 06 06:15:49.937976 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.937948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:49.976086 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.976042    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:49.976291 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:49.976270    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" podUID="539704ca-b437-4147-ace2-a7536983e684" containerName="payload-pre-processing" containerID="cri-o://c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6" gracePeriod=30
Sep 06 06:15:50.038349 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.038311    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:50.129984 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.129907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:50.189664 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.189572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:50.230410 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.230384    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:50.240515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.240494    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:50.338902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.338868    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs\") pod \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") "
Sep 06 06:15:50.338902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.338905    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca\") pod \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") "
Sep 06 06:15:50.339169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.338947    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58rfh\" (UniqueName: \"kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh\") pod \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") "
Sep 06 06:15:50.339169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.338971    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qdv2\" (UniqueName: \"kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2\") pod \"539704ca-b437-4147-ace2-a7536983e684\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") "
Sep 06 06:15:50.339169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.339000    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume\") pod \"539704ca-b437-4147-ace2-a7536983e684\" (UID: \"539704ca-b437-4147-ace2-a7536983e684\") "
Sep 06 06:15:50.339169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.339126    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") pod \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\" (UID: \"bbda4532-b0df-4a1b-b561-a3a5167a26b5\") "
Sep 06 06:15:50.339475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.339434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bbda4532-b0df-4a1b-b561-a3a5167a26b5" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:50.339475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.339451    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "539704ca-b437-4147-ace2-a7536983e684" (UID: "539704ca-b437-4147-ace2-a7536983e684"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:15:50.341324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.341292    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bbda4532-b0df-4a1b-b561-a3a5167a26b5" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:50.341423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.341302    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bbda4532-b0df-4a1b-b561-a3a5167a26b5" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:15:50.341423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.341326    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh" (OuterVolumeSpecName: "kube-api-access-58rfh") pod "bbda4532-b0df-4a1b-b561-a3a5167a26b5" (UID: "bbda4532-b0df-4a1b-b561-a3a5167a26b5"). InnerVolumeSpecName "kube-api-access-58rfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:50.341423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.341308    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2" (OuterVolumeSpecName: "kube-api-access-2qdv2") pod "539704ca-b437-4147-ace2-a7536983e684" (UID: "539704ca-b437-4147-ace2-a7536983e684"). InnerVolumeSpecName "kube-api-access-2qdv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.439965    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58rfh\" (UniqueName: \"kubernetes.io/projected/bbda4532-b0df-4a1b-b561-a3a5167a26b5-kube-api-access-58rfh\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.439991    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qdv2\" (UniqueName: \"kubernetes.io/projected/539704ca-b437-4147-ace2-a7536983e684-kube-api-access-2qdv2\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.440001    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/539704ca-b437-4147-ace2-a7536983e684-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.440012    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.440021    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbda4532-b0df-4a1b-b561-a3a5167a26b5-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.440041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.440028    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbda4532-b0df-4a1b-b561-a3a5167a26b5-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:15:50.470443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.470414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:50.602605 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.602570    2567 generic.go:356] "Generic (PLEG): container finished" podID="539704ca-b437-4147-ace2-a7536983e684" containerID="c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6" exitCode=0
Sep 06 06:15:50.602793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.602614    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"
Sep 06 06:15:50.602793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.602658    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" event={"ID":"539704ca-b437-4147-ace2-a7536983e684","Type":"ContainerDied","Data":"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"}
Sep 06 06:15:50.602793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.602713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5" event={"ID":"539704ca-b437-4147-ace2-a7536983e684","Type":"ContainerDied","Data":"a5796733ed0c7fc8374d7a40da17514cb851308ed1540bb809622f74510f95f1"}
Sep 06 06:15:50.602793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.602741    2567 scope.go:117] "RemoveContainer" containerID="c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"
Sep 06 06:15:50.604238 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.604214    2567 generic.go:356] "Generic (PLEG): container finished" podID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" containerID="2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75" exitCode=0
Sep 06 06:15:50.604329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.604264    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"
Sep 06 06:15:50.604329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.604270    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" event={"ID":"bbda4532-b0df-4a1b-b561-a3a5167a26b5","Type":"ContainerDied","Data":"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"}
Sep 06 06:15:50.604329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.604320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd" event={"ID":"bbda4532-b0df-4a1b-b561-a3a5167a26b5","Type":"ContainerDied","Data":"6ed35ed6e3e55f399322fda2f904358e717160387d065594e99b704fc53ab445"}
Sep 06 06:15:50.611630 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.611614    2567 scope.go:117] "RemoveContainer" containerID="c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"
Sep 06 06:15:50.611931 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:50.611913    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6\": container with ID starting with c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6 not found: ID does not exist" containerID="c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"
Sep 06 06:15:50.612005 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.611943    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6"} err="failed to get container status \"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6\": rpc error: code = NotFound desc = could not find container \"c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6\": container with ID starting with c203a349307eef287f8109b1da0681cbead3996b7c9b338e2349428c027f3ff6 not found: ID does not exist"
Sep 06 06:15:50.612005 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.611964    2567 scope.go:117] "RemoveContainer" containerID="2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"
Sep 06 06:15:50.620489 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.620470    2567 scope.go:117] "RemoveContainer" containerID="2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"
Sep 06 06:15:50.620749 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:15:50.620727    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75\": container with ID starting with 2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75 not found: ID does not exist" containerID="2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"
Sep 06 06:15:50.620832 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.620752    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75"} err="failed to get container status \"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75\": rpc error: code = NotFound desc = could not find container \"2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75\": container with ID starting with 2ac227689877d0200d282e6b3141bf81d63be8e5568a77ebe2a2be816140bf75 not found: ID does not exist"
Sep 06 06:15:50.628812 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.628792    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:50.630795 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.630775    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fc95779d-6dc9fcd786-582q5"]
Sep 06 06:15:50.642946 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.642921    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:50.645918 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.645897    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc95779d-7dbf9cb55b-nccwd"]
Sep 06 06:15:50.786122 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.786006    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="539704ca-b437-4147-ace2-a7536983e684" path="/var/lib/kubelet/pods/539704ca-b437-4147-ace2-a7536983e684/volumes"
Sep 06 06:15:50.786501 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:50.786480    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" path="/var/lib/kubelet/pods/bbda4532-b0df-4a1b-b561-a3a5167a26b5/volumes"
Sep 06 06:15:52.320472 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:52.320437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:52.387409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:52.387373    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:52.581259 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:52.581176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:53.113423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.113383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:53.391517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391443    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391782    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b2c4d6-632d-4137-886f-970d10618a66" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391794    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b2c4d6-632d-4137-886f-970d10618a66" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391803    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391808    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391817    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7ca98f-151f-4e34-8b28-b37e2318126b" containerName="payload-processing"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391825    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7ca98f-151f-4e34-8b28-b37e2318126b" containerName="payload-processing"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391840    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391846    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" containerName="maas-api"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391852    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" containerName="main"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391857    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" containerName="main"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391865    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e6567d6-997b-4d57-84b4-74e2126266a5" containerName="payload-pre-processing"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391870    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e6567d6-997b-4d57-84b4-74e2126266a5" containerName="payload-pre-processing"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391876    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="539704ca-b437-4147-ace2-a7536983e684" containerName="payload-pre-processing"
Sep 06 06:15:53.391874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391881    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="539704ca-b437-4147-ace2-a7536983e684" containerName="payload-pre-processing"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391941    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b42528-4b57-4e46-88bc-81834e8652ce" containerName="maas-api"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391950    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="64b840e5-5d35-4271-8a30-c3c2107fcf3f" containerName="main"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391956    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b2c4d6-632d-4137-886f-970d10618a66" containerName="maas-api"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391962    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="539704ca-b437-4147-ace2-a7536983e684" containerName="payload-pre-processing"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391967    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbda4532-b0df-4a1b-b561-a3a5167a26b5" containerName="maas-api"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391972    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e6567d6-997b-4d57-84b4-74e2126266a5" containerName="payload-pre-processing"
Sep 06 06:15:53.392409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.391981    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7ca98f-151f-4e34-8b28-b37e2318126b" containerName="payload-processing"
Sep 06 06:15:53.396627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.396607    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.399356 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.399334    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-429831-openshift-default-dockercfg-ssxwm\""
Sep 06 06:15:53.406784 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.406757    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:15:53.423138 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.423109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:53.466420 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466458    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6n47\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466557 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466582    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466657    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.466697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.466685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567363    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6n47\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567914 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567761    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567974 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.567974 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.567923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.568089 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.568004    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.568302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.568284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.569638 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.569618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.569990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.569969    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.578425 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.578400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.578870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.578845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6n47\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47\") pod \"e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") " pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.708043 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.707946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:15:53.786011 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.785973    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:53.985387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:53.985363    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:15:53.986502 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:15:53.986476    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721aa5f0_1d56_4a34_aef7_534353ae35b3.slice/crio-5f2c18c770a2eb7423e79ec56a9d6d68d18ff2ca4e30652a63b562d22ce9cfbd WatchSource:0}: Error finding container 5f2c18c770a2eb7423e79ec56a9d6d68d18ff2ca4e30652a63b562d22ce9cfbd: Status 404 returned error can't find the container with id 5f2c18c770a2eb7423e79ec56a9d6d68d18ff2ca4e30652a63b562d22ce9cfbd
Sep 06 06:15:54.627959 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:54.627924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerStarted","Data":"5f2c18c770a2eb7423e79ec56a9d6d68d18ff2ca4e30652a63b562d22ce9cfbd"}
Sep 06 06:15:55.996366 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:55.996327    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:56.096623 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:56.096591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:56.258606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:56.258519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:56.575037 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:56.575002    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:56.676403 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:56.676372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:56.784102 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:56.784049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:15:58.730877 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:15:58.730837    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:02.540836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:02.540797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:02.600786 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:02.600748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:02.681465 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:02.681430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:02.781790 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:02.781752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:02.881792 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:02.881758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:03.446739 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:03.446701    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:07.796070 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:07.796031    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:07.796369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:07.796115    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:07.796369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:07.796144    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:08.372617 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.372581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:08.471384 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.471347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:08.703624 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.703522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerStarted","Data":"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"}
Sep 06 06:16:08.708535 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.708494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:08.708535 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.708539    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:08.713429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.713404    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:08.724852 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:08.724806    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podStartSLOduration=1.9174192300000001 podStartE2EDuration="15.72479271s" totalImagesPullingTime="13.80737348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:08.722519584 +0000 UTC m=+1322.426072322" watchObservedRunningTime="2026-09-06 06:16:08.72479271 +0000 UTC m=+1322.428345520"
Sep 06 06:16:09.273860 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:09.273821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:09.576869 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:09.576835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:09.709704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:09.709663    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:10.002414 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:10.002326    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:10.757743 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:10.757702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:10.848249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:10.848206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:11.876498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.876461    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:16:11.880189 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.880160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:11.882981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.882958    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-429831-serving-cert\""
Sep 06 06:16:11.892598 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.892569    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:16:11.905698 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.905664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:11.905698 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.905702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:11.905912 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.905727    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh2km\" (UniqueName: \"kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:11.905912 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.905800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:11.916568 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:11.916514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.007241 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.007204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.007241 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.007245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.007513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.007274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh2km\" (UniqueName: \"kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.007513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.007306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.007513 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:12.007377    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-serving-cert: secret "maas-api-e2e-shared-a-w3-429831-serving-cert" not found
Sep 06 06:16:12.007513 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:12.007447    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls podName:2761ca3c-378c-4c78-b2c7-692ac3b01e6e nodeName:}" failed. No retries permitted until 2026-09-06 06:16:12.507427346 +0000 UTC m=+1326.210980067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls") pod "maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" (UID: "2761ca3c-378c-4c78-b2c7-692ac3b01e6e") : secret "maas-api-e2e-shared-a-w3-429831-serving-cert" not found
Sep 06 06:16:12.007987 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.007961    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.009848 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.009827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.019191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.019170    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh2km\" (UniqueName: \"kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.020980 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.020956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.119263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.119228    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.369724 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.369671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.473097 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.473026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.498510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.498482    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:16:12.501950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.501934    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.505135 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.505087    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-429831\""
Sep 06 06:16:12.505259 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.505239    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-429831-dockercfg-4plqh\""
Sep 06 06:16:12.512773 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.512753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.515109 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.515084    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.528603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.528574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:16:12.539816 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.539791    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:16:12.543588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.543565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.561974 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.561945    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:16:12.613958 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.613923    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.614153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.613993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4lwj\" (UniqueName: \"kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.614153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.614043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6qf\" (UniqueName: \"kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.614280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.614151    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.677386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.677283    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.715649 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.715614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.716868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.716845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.717779 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.717753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4lwj\" (UniqueName: \"kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.718224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.718206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6qf\" (UniqueName: \"kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.718667 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.716675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.718944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.717697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.728170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.728143    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4lwj\" (UniqueName: \"kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj\") pod \"payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:12.728265 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.728197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6qf\" (UniqueName: \"kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf\") pod \"payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.759945 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.759903    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:12.793386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.793362    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:12.811965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.811941    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:12.854813 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:12.853760    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:13.077026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.076998    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:16:13.079197 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:13.079168    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2761ca3c_378c_4c78_b2c7_692ac3b01e6e.slice/crio-7ca23479943f61e78c121982bd64cc5ae8e6f4cc0c190c35258c147e0315a886 WatchSource:0}: Error finding container 7ca23479943f61e78c121982bd64cc5ae8e6f4cc0c190c35258c147e0315a886: Status 404 returned error can't find the container with id 7ca23479943f61e78c121982bd64cc5ae8e6f4cc0c190c35258c147e0315a886
Sep 06 06:16:13.080648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.080629    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 06:16:13.123517 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:13.123210    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66728df_2ebd_48e8_8c32_607dfc52fcbb.slice/crio-06c05544d3d99a2f07e160240ea05bd3e7b1f3b7d5b471b757404918d44c3afc WatchSource:0}: Error finding container 06c05544d3d99a2f07e160240ea05bd3e7b1f3b7d5b471b757404918d44c3afc: Status 404 returned error can't find the container with id 06c05544d3d99a2f07e160240ea05bd3e7b1f3b7d5b471b757404918d44c3afc
Sep 06 06:16:13.126836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.126801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:16:13.146925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.146899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:16:13.147817 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:13.147788    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a6afbc_7263_4294_8cb8_62a42bc405c1.slice/crio-9d6caa2868fea957ac1d72ca5467901d5de03771fae92fc0e1d0079af7fa6f94 WatchSource:0}: Error finding container 9d6caa2868fea957ac1d72ca5467901d5de03771fae92fc0e1d0079af7fa6f94: Status 404 returned error can't find the container with id 9d6caa2868fea957ac1d72ca5467901d5de03771fae92fc0e1d0079af7fa6f94
Sep 06 06:16:13.732118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.732014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" event={"ID":"58a6afbc-7263-4294-8cb8-62a42bc405c1","Type":"ContainerStarted","Data":"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"}
Sep 06 06:16:13.732118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.732102    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:13.732350 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.732129    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" event={"ID":"58a6afbc-7263-4294-8cb8-62a42bc405c1","Type":"ContainerStarted","Data":"9d6caa2868fea957ac1d72ca5467901d5de03771fae92fc0e1d0079af7fa6f94"}
Sep 06 06:16:13.734294 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.734253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" event={"ID":"e66728df-2ebd-48e8-8c32-607dfc52fcbb","Type":"ContainerStarted","Data":"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"}
Sep 06 06:16:13.734437 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.734316    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" event={"ID":"e66728df-2ebd-48e8-8c32-607dfc52fcbb","Type":"ContainerStarted","Data":"06c05544d3d99a2f07e160240ea05bd3e7b1f3b7d5b471b757404918d44c3afc"}
Sep 06 06:16:13.734437 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.734349    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:13.736136 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.736111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" event={"ID":"2761ca3c-378c-4c78-b2c7-692ac3b01e6e","Type":"ContainerStarted","Data":"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"}
Sep 06 06:16:13.736244 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.736153    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" event={"ID":"2761ca3c-378c-4c78-b2c7-692ac3b01e6e","Type":"ContainerStarted","Data":"7ca23479943f61e78c121982bd64cc5ae8e6f4cc0c190c35258c147e0315a886"}
Sep 06 06:16:13.736244 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.736238    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:13.748233 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.748160    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" podStartSLOduration=1.7481382380000001 podStartE2EDuration="1.748138238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:13.747539091 +0000 UTC m=+1327.451091831" watchObservedRunningTime="2026-09-06 06:16:13.748138238 +0000 UTC m=+1327.451690975"
Sep 06 06:16:13.763685 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.763631    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" podStartSLOduration=1.763614557 podStartE2EDuration="1.763614557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:13.761997599 +0000 UTC m=+1327.465550347" watchObservedRunningTime="2026-09-06 06:16:13.763614557 +0000 UTC m=+1327.467167291"
Sep 06 06:16:13.780035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:13.779973    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" podStartSLOduration=2.403168244 podStartE2EDuration="2.779954561s" totalImagesPullingTime="376.786317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:13.777706328 +0000 UTC m=+1327.481259066" watchObservedRunningTime="2026-09-06 06:16:13.779954561 +0000 UTC m=+1327.483507298"
Sep 06 06:16:14.004695 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.004592    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:16:14.007838 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.007813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.010541 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.010511    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-340e00-serving-cert\""
Sep 06 06:16:14.017042 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.017010    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:16:14.034121 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.034026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttbj\" (UniqueName: \"kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.034296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.034205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.034399 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.034363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.034515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.034474    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.041925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.041890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:14.096208 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.096166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:14.135648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.135606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttbj\" (UniqueName: \"kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.135868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.135700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.135868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.135792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.135868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.135846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.136017 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:14.135954    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-serving-cert: secret "maas-api-e2e-shared-a-w2-340e00-serving-cert" not found
Sep 06 06:16:14.136096 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:14.136018    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls podName:10ca52c5-3600-4d91-8250-8757e74ad2b9 nodeName:}" failed. No retries permitted until 2026-09-06 06:16:14.635997043 +0000 UTC m=+1328.339549771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls") pod "maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9") : secret "maas-api-e2e-shared-a-w2-340e00-serving-cert" not found
Sep 06 06:16:14.136625 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.136597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.138690 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.138658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.144600 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.144572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttbj\" (UniqueName: \"kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.503799 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.503758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:14.604245 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.604210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:14.642389 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.642350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:14.642569 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:14.642500    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-serving-cert: secret "maas-api-e2e-shared-a-w2-340e00-serving-cert" not found
Sep 06 06:16:14.642613 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:14.642571    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls podName:10ca52c5-3600-4d91-8250-8757e74ad2b9 nodeName:}" failed. No retries permitted until 2026-09-06 06:16:15.642554446 +0000 UTC m=+1329.346107160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls") pod "maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9") : secret "maas-api-e2e-shared-a-w2-340e00-serving-cert" not found
Sep 06 06:16:14.707304 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:14.707270    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:15.651884 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:15.651848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:15.654296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:15.654277    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:15.836386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:15.836342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:16.108205 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:16.108177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:16:16.110110 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:16.110081    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ca52c5_3600_4d91_8250_8757e74ad2b9.slice/crio-7996249154a9828d8e7b4c752c8bfd6d181ea2d34419dcfddbacfeff806eb786 WatchSource:0}: Error finding container 7996249154a9828d8e7b4c752c8bfd6d181ea2d34419dcfddbacfeff806eb786: Status 404 returned error can't find the container with id 7996249154a9828d8e7b4c752c8bfd6d181ea2d34419dcfddbacfeff806eb786
Sep 06 06:16:16.755276 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:16.755242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" event={"ID":"10ca52c5-3600-4d91-8250-8757e74ad2b9","Type":"ContainerStarted","Data":"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"}
Sep 06 06:16:16.755704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:16.755297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" event={"ID":"10ca52c5-3600-4d91-8250-8757e74ad2b9","Type":"ContainerStarted","Data":"7996249154a9828d8e7b4c752c8bfd6d181ea2d34419dcfddbacfeff806eb786"}
Sep 06 06:16:16.755704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:16.755390    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:16.770862 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:16.770750    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" podStartSLOduration=3.365560445 podStartE2EDuration="3.770729576s" totalImagesPullingTime="405.169131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:16.770143222 +0000 UTC m=+1330.473695959" watchObservedRunningTime="2026-09-06 06:16:16.770729576 +0000 UTC m=+1330.474282313"
Sep 06 06:16:19.751271 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:19.751238    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:16:22.769770 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:22.769737    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:16:24.743276 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:24.743232    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:16:24.743883 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:24.743517    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:16:25.750227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:25.750187    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 06:16:25.750634 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:25.750246    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 06:16:27.829984 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.829949    2567 generic.go:356] "Generic (PLEG): container finished" podID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerID="ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53" exitCode=0
Sep 06 06:16:27.830418 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.830022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerDied","Data":"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"}
Sep 06 06:16:27.830475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.830448    2567 scope.go:117] "RemoveContainer" containerID="ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"
Sep 06 06:16:27.832488 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.832459    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:27.832565 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.832526    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:27.832565 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:27.832557    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 06 06:16:28.841472 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:28.841428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerStarted","Data":"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"}
Sep 06 06:16:29.708253 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:29.708212    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:29.708452 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:29.708426    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 06 06:16:29.708510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:29.708486    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 06 06:16:30.708445 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:30.708404    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 06 06:16:30.708858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:30.708471    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 06 06:16:31.708897 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:31.708858    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 06 06:16:31.709361 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:31.708927    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 06 06:16:32.708730 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:32.708691    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 06 06:16:32.708907 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:32.708754    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 06 06:16:32.999295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:32.999180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:33.708684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:33.708650    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 06 06:16:33.708929 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:33.708704    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 06 06:16:34.776039 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:34.776004    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:34.776431 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:34.776395    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:34.777046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:34.777027    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:16:34.923159 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:34.923114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:35.608458 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:35.608418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:35.930689 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:35.930595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:36.253187 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:36.253092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:36.349933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:36.349896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:37.562672 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:37.562640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:37.873174 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:37.873092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:38.159041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:38.158965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:38.447211 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:38.447134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:39.684080 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:39.684023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:39.783928 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:39.783894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:40.789863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.789829    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:16:40.793600 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.793582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.795990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.795971    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-dc97f9-serving-cert\""
Sep 06 06:16:40.803929 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.803898    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:16:40.885153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.885119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:40.888301 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.888270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4wcg\" (UniqueName: \"kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.888457 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.888332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.888457 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.888391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.888457 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.888448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.986121 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.986089    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:40.989246 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.989223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.989344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.989264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.989344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.989304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.989344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.989336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4wcg\" (UniqueName: \"kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.989445 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:40.989416    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-serving-cert: secret "maas-api-e2e-shared-b-w3-dc97f9-serving-cert" not found
Sep 06 06:16:40.989531 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:40.989519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls podName:b7346d10-91b5-4170-b050-43ff825e95a5 nodeName:}" failed. No retries permitted until 2026-09-06 06:16:41.489496702 +0000 UTC m=+1355.193049430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls") pod "maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" (UID: "b7346d10-91b5-4170-b050-43ff825e95a5") : secret "maas-api-e2e-shared-b-w3-dc97f9-serving-cert" not found
Sep 06 06:16:40.990021 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.990002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.991702 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.991682    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:40.997751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:40.997727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4wcg\" (UniqueName: \"kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:41.211196 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.211167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:41.395963 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.395928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:41.493895 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.493797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:41.495921 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.495884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:41.496508 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.496489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:41.704533 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.704496    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:41.936417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.936379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:41.980365 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:41.980334    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:16:41.981528 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:41.981499    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7346d10_91b5_4170_b050_43ff825e95a5.slice/crio-fcd3fe64c95072d4a0aebb0a12002c7324598de9820af873e703be760905f84c WatchSource:0}: Error finding container fcd3fe64c95072d4a0aebb0a12002c7324598de9820af873e703be760905f84c: Status 404 returned error can't find the container with id fcd3fe64c95072d4a0aebb0a12002c7324598de9820af873e703be760905f84c
Sep 06 06:16:42.026973 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:42.026908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:42.920384 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:42.920344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" event={"ID":"b7346d10-91b5-4170-b050-43ff825e95a5","Type":"ContainerStarted","Data":"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"}
Sep 06 06:16:42.920586 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:42.920404    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" event={"ID":"b7346d10-91b5-4170-b050-43ff825e95a5","Type":"ContainerStarted","Data":"fcd3fe64c95072d4a0aebb0a12002c7324598de9820af873e703be760905f84c"}
Sep 06 06:16:42.920586 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:42.920425    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:42.965369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:42.965315    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" podStartSLOduration=2.665261575 podStartE2EDuration="2.965299986s" totalImagesPullingTime="300.038411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:42.963173144 +0000 UTC m=+1356.666725893" watchObservedRunningTime="2026-09-06 06:16:42.965299986 +0000 UTC m=+1356.668852758"
Sep 06 06:16:43.091217 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.091184    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:16:43.095343 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.095320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.097840 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.097816    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-b7c728-serving-cert\""
Sep 06 06:16:43.104145 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.104115    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:16:43.125233 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.125205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:43.183260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.183176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:43.210362 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.210328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.210539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.210392    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.210539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.210416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbldz\" (UniqueName: \"kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.210539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.210482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.311793 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.311753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.311985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.311871    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.311985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.311909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.311985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.311932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbldz\" (UniqueName: \"kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.312269 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:43.312022    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-serving-cert: secret "maas-api-e2e-shared-b-w2-b7c728-serving-cert" not found
Sep 06 06:16:43.312269 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:16:43.312127    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls podName:b2c43099-4f68-4d0d-bed8-c0650a77e6f5 nodeName:}" failed. No retries permitted until 2026-09-06 06:16:43.812105713 +0000 UTC m=+1357.515658445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls") pod "maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" (UID: "b2c43099-4f68-4d0d-bed8-c0650a77e6f5") : secret "maas-api-e2e-shared-b-w2-b7c728-serving-cert" not found
Sep 06 06:16:43.312573 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.312555    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.314517 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.314495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.321029 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.321005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbldz\" (UniqueName: \"kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.431165 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.431133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:43.699261 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.699227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:43.793591 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.793529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:43.816689 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.816655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:43.819194 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:43.819168    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:44.009280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.009196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:44.289507 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.289480    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:16:44.290652 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:16:44.290629    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c43099_4f68_4d0d_bed8_c0650a77e6f5.slice/crio-10f1737f1352004ebe2469e5d746f4fe87ed32ed6a031e9e9728a90dcc162ec5 WatchSource:0}: Error finding container 10f1737f1352004ebe2469e5d746f4fe87ed32ed6a031e9e9728a90dcc162ec5: Status 404 returned error can't find the container with id 10f1737f1352004ebe2469e5d746f4fe87ed32ed6a031e9e9728a90dcc162ec5
Sep 06 06:16:44.935515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.935436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" event={"ID":"b2c43099-4f68-4d0d-bed8-c0650a77e6f5","Type":"ContainerStarted","Data":"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"}
Sep 06 06:16:44.935515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.935497    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" event={"ID":"b2c43099-4f68-4d0d-bed8-c0650a77e6f5","Type":"ContainerStarted","Data":"10f1737f1352004ebe2469e5d746f4fe87ed32ed6a031e9e9728a90dcc162ec5"}
Sep 06 06:16:44.935691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.935523    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:44.952165 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:44.952117    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" podStartSLOduration=1.652390279 podStartE2EDuration="1.952102168s" totalImagesPullingTime="299.711889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:16:44.951050012 +0000 UTC m=+1358.654602748" watchObservedRunningTime="2026-09-06 06:16:44.952102168 +0000 UTC m=+1358.655654901"
Sep 06 06:16:48.934943 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:48.934908    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:16:50.946744 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:50.946715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:16:58.066044 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:58.066008    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:58.115227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:58.115188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:16:58.801196 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:16:58.801156    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:01.899234 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:01.899198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:04.261163 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:04.261125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:04.620804 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:04.620765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:06.673041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:06.672977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:12.427653 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:12.427607    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:19.856227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:19.856191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:20.144313 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:20.144188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:20.386315 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:20.386277    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:23.086521 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:23.086472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:23.184611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:23.184576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.188822 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.188779    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:17:24.194156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.194128    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.196614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.196591    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dd8c231a-serving-cert\""
Sep 06 06:17:24.202116 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.202042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:17:24.221033 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.221001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.286850 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.286815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqp4s\" (UniqueName: \"kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.287047 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.286868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.287047 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.286977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.287047 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.287020    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.322248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.322212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.388041 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.388004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqp4s\" (UniqueName: \"kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.388288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.388047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.388288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.388128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.388288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.388157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.388288 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:17:24.388257    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-serving-cert: secret "maas-api-e2e-mt-dd8c231a-serving-cert" not found
Sep 06 06:17:24.388498 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:17:24.388328    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls podName:7cdbfe90-195f-4725-9be0-1b997d624edc nodeName:}" failed. No retries permitted until 2026-09-06 06:17:24.88831143 +0000 UTC m=+1398.591864144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls") pod "maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" (UID: "7cdbfe90-195f-4725-9be0-1b997d624edc") : secret "maas-api-e2e-mt-dd8c231a-serving-cert" not found
Sep 06 06:17:24.388950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.388924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.390744 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.390719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.396378 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.396356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqp4s\" (UniqueName: \"kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.531880 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.531773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.696649 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.696616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.794874 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.794790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:24.858161 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.858121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:17:24.861736 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.861715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:24.865687 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.865672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dd8c231a-dockercfg-h7ghw\""
Sep 06 06:17:24.866001 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.865985    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dd8c231a\""
Sep 06 06:17:24.877554 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.877531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:17:24.893387 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.893361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.895727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.895709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") pod \"maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:24.994299 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.994249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:24.994477 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:24.994370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jscq\" (UniqueName: \"kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.094925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.094888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.095116 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.094947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jscq\" (UniqueName: \"kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.095646 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.095624    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.102910 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.102879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jscq\" (UniqueName: \"kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq\") pod \"payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") " pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.108749 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.108727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:25.172441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.172408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:25.398703 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.398679    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:17:25.400817 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:17:25.400792    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cdbfe90_195f_4725_9be0_1b997d624edc.slice/crio-486978bf69145ac128ac5f08c0e0327d42a4276d5630e318c7fb1c937d3d00e2 WatchSource:0}: Error finding container 486978bf69145ac128ac5f08c0e0327d42a4276d5630e318c7fb1c937d3d00e2: Status 404 returned error can't find the container with id 486978bf69145ac128ac5f08c0e0327d42a4276d5630e318c7fb1c937d3d00e2
Sep 06 06:17:25.472463 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:25.472426    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:17:26.161455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.161420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" event={"ID":"7cdbfe90-195f-4725-9be0-1b997d624edc","Type":"ContainerStarted","Data":"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"}
Sep 06 06:17:26.161656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.161521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" event={"ID":"7cdbfe90-195f-4725-9be0-1b997d624edc","Type":"ContainerStarted","Data":"486978bf69145ac128ac5f08c0e0327d42a4276d5630e318c7fb1c937d3d00e2"}
Sep 06 06:17:26.161656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.161556    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:26.163036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.163013    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" event={"ID":"ee75e129-62bd-4069-932a-41d31ca90960","Type":"ContainerStarted","Data":"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"}
Sep 06 06:17:26.163169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.163050    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" event={"ID":"ee75e129-62bd-4069-932a-41d31ca90960","Type":"ContainerStarted","Data":"de595ef93fd64e67346ed5f7e00eac5b39f1df37f3ac2d3260be523a30305c2a"}
Sep 06 06:17:26.163237 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.163173    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:26.179423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.179358    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" podStartSLOduration=1.850860021 podStartE2EDuration="2.179344227s" totalImagesPullingTime="328.484206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:17:26.176250043 +0000 UTC m=+1399.879802780" watchObservedRunningTime="2026-09-06 06:17:26.179344227 +0000 UTC m=+1399.882896987"
Sep 06 06:17:26.191709 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:26.191662    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" podStartSLOduration=2.191647995 podStartE2EDuration="2.191647995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:17:26.190703204 +0000 UTC m=+1399.894255951" watchObservedRunningTime="2026-09-06 06:17:26.191647995 +0000 UTC m=+1399.895200730"
Sep 06 06:17:32.173867 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:32.173835    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:17:37.170567 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:37.170536    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:17:43.324454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:43.324415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:17:47.917743 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:17:47.917701    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:18:20.006636 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:18:20.006599    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:18:24.725036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:18:24.725001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:18:52.902775 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:18:52.902737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:18:55.366237 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:18:55.366197    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:18:55.416247 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:18:55.416214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:08.306348 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.306310    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:19:08.306811 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.306614    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" podUID="7cdbfe90-195f-4725-9be0-1b997d624edc" containerName="maas-api" containerID="cri-o://e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10" gracePeriod=30
Sep 06 06:19:08.330899 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.330858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:08.355580 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.355519    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:19:08.355852 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.355817    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" podUID="ee75e129-62bd-4069-932a-41d31ca90960" containerName="payload-processing" containerID="cri-o://63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64" gracePeriod=30
Sep 06 06:19:08.396487 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.396436    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:08.496789 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.496682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:08.550127 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.550089    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:08.619915 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.619889    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:19:08.653342 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.653319    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:19:08.723662 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.723629    2567 generic.go:356] "Generic (PLEG): container finished" podID="7cdbfe90-195f-4725-9be0-1b997d624edc" containerID="e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10" exitCode=0
Sep 06 06:19:08.723830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.723663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" event={"ID":"7cdbfe90-195f-4725-9be0-1b997d624edc","Type":"ContainerDied","Data":"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"}
Sep 06 06:19:08.723830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.723696    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"
Sep 06 06:19:08.723830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.723723    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7" event={"ID":"7cdbfe90-195f-4725-9be0-1b997d624edc","Type":"ContainerDied","Data":"486978bf69145ac128ac5f08c0e0327d42a4276d5630e318c7fb1c937d3d00e2"}
Sep 06 06:19:08.723830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.723751    2567 scope.go:117] "RemoveContainer" containerID="e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"
Sep 06 06:19:08.725110 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.725089    2567 generic.go:356] "Generic (PLEG): container finished" podID="ee75e129-62bd-4069-932a-41d31ca90960" containerID="63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64" exitCode=0
Sep 06 06:19:08.725218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.725124    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"
Sep 06 06:19:08.725218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.725152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" event={"ID":"ee75e129-62bd-4069-932a-41d31ca90960","Type":"ContainerDied","Data":"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"}
Sep 06 06:19:08.725218 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.725193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt" event={"ID":"ee75e129-62bd-4069-932a-41d31ca90960","Type":"ContainerDied","Data":"de595ef93fd64e67346ed5f7e00eac5b39f1df37f3ac2d3260be523a30305c2a"}
Sep 06 06:19:08.734604 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.734583    2567 scope.go:117] "RemoveContainer" containerID="e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"
Sep 06 06:19:08.734868 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:19:08.734851    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10\": container with ID starting with e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10 not found: ID does not exist" containerID="e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"
Sep 06 06:19:08.734959 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.734874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10"} err="failed to get container status \"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10\": rpc error: code = NotFound desc = could not find container \"e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10\": container with ID starting with e5b141525f9068593e7bc9af924e486b48b30185ae62b706621397c3f7d38c10 not found: ID does not exist"
Sep 06 06:19:08.734959 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.734890    2567 scope.go:117] "RemoveContainer" containerID="63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"
Sep 06 06:19:08.742785 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.742769    2567 scope.go:117] "RemoveContainer" containerID="63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"
Sep 06 06:19:08.743055 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:19:08.743034    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64\": container with ID starting with 63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64 not found: ID does not exist" containerID="63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"
Sep 06 06:19:08.743147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.743078    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64"} err="failed to get container status \"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64\": rpc error: code = NotFound desc = could not find container \"63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64\": container with ID starting with 63e3deb3843a6220f24f7de27b2d86bae5eece784db0d06d922e18f0dd2e9d64 not found: ID does not exist"
Sep 06 06:19:08.746413 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca\") pod \"7cdbfe90-195f-4725-9be0-1b997d624edc\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") "
Sep 06 06:19:08.746513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746426    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") pod \"7cdbfe90-195f-4725-9be0-1b997d624edc\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") "
Sep 06 06:19:08.746513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746450    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs\") pod \"7cdbfe90-195f-4725-9be0-1b997d624edc\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") "
Sep 06 06:19:08.746513 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746492    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jscq\" (UniqueName: \"kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq\") pod \"ee75e129-62bd-4069-932a-41d31ca90960\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") "
Sep 06 06:19:08.746678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746567    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume\") pod \"ee75e129-62bd-4069-932a-41d31ca90960\" (UID: \"ee75e129-62bd-4069-932a-41d31ca90960\") "
Sep 06 06:19:08.746678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746618    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqp4s\" (UniqueName: \"kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s\") pod \"7cdbfe90-195f-4725-9be0-1b997d624edc\" (UID: \"7cdbfe90-195f-4725-9be0-1b997d624edc\") "
Sep 06 06:19:08.746850 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.746798    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7cdbfe90-195f-4725-9be0-1b997d624edc" (UID: "7cdbfe90-195f-4725-9be0-1b997d624edc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:19:08.747231 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.747194    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee75e129-62bd-4069-932a-41d31ca90960" (UID: "ee75e129-62bd-4069-932a-41d31ca90960"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:19:08.748597 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.748572    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cdbfe90-195f-4725-9be0-1b997d624edc-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:08.748686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.748603    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee75e129-62bd-4069-932a-41d31ca90960-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:08.749112 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.749088    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7cdbfe90-195f-4725-9be0-1b997d624edc" (UID: "7cdbfe90-195f-4725-9be0-1b997d624edc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:19:08.749196 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.749148    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s" (OuterVolumeSpecName: "kube-api-access-hqp4s") pod "7cdbfe90-195f-4725-9be0-1b997d624edc" (UID: "7cdbfe90-195f-4725-9be0-1b997d624edc"). InnerVolumeSpecName "kube-api-access-hqp4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:19:08.749346 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.749324    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq" (OuterVolumeSpecName: "kube-api-access-6jscq") pod "ee75e129-62bd-4069-932a-41d31ca90960" (UID: "ee75e129-62bd-4069-932a-41d31ca90960"). InnerVolumeSpecName "kube-api-access-6jscq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:19:08.749584 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.749567    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7cdbfe90-195f-4725-9be0-1b997d624edc" (UID: "7cdbfe90-195f-4725-9be0-1b997d624edc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:19:08.849428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.849358    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqp4s\" (UniqueName: \"kubernetes.io/projected/7cdbfe90-195f-4725-9be0-1b997d624edc-kube-api-access-hqp4s\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:08.849428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.849386    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:08.849428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.849395    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cdbfe90-195f-4725-9be0-1b997d624edc-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:08.849428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:08.849403    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jscq\" (UniqueName: \"kubernetes.io/projected/ee75e129-62bd-4069-932a-41d31ca90960-kube-api-access-6jscq\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:19:09.040179 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:09.040142    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:19:09.045202 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:09.045178    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dd8c231a-6c7dfb889d-5whw7"]
Sep 06 06:19:09.054980 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:09.054956    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:19:09.058866 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:09.058843    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dd8c231a-74c96b869c-j8dbt"]
Sep 06 06:19:10.787303 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:10.787267    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cdbfe90-195f-4725-9be0-1b997d624edc" path="/var/lib/kubelet/pods/7cdbfe90-195f-4725-9be0-1b997d624edc/volumes"
Sep 06 06:19:10.787689 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:10.787636    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee75e129-62bd-4069-932a-41d31ca90960" path="/var/lib/kubelet/pods/ee75e129-62bd-4069-932a-41d31ca90960/volumes"
Sep 06 06:19:11.543981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:11.543940    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:16.569393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:16.569351    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:52.713755 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:52.713713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:53.406549 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:53.406492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:58.607993 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:58.607942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:59.228747 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:59.228705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:59.329263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:59.329217    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:59.464756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:59.464720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:59.622421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:59.622386    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:19:59.877143 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:19:59.877025    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:00.641412 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:00.641376    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:00.739114 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:00.739043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:00.875862 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:00.875829    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:01.416466 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:01.416429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:01.515353 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:01.515319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:01.615026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:01.614989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:02.794706 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:02.794669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:08.480575 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:08.480537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:08.535792 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:08.535745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:09.126504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:09.126470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:14.726971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:14.726933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:14.787019 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:14.786980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:14.868307 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:14.868266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:14.968537 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:14.968505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:25.400252 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:25.400216    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:25.624202 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:25.624167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:32.534162 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:32.534118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:32.590375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:32.590346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:40.397808 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:40.397769    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:46.597598 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:46.597555    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:52.500715 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:52.500678    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:52.601318 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:52.601282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:52.828671 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:52.828632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:53.117483 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:53.117401    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:53.623577 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:53.623544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:56.308558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:56.308523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:56.409499 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:56.409461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:57.497018 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.496982    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:20:57.497585 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497563    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cdbfe90-195f-4725-9be0-1b997d624edc" containerName="maas-api"
Sep 06 06:20:57.497585 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497583    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cdbfe90-195f-4725-9be0-1b997d624edc" containerName="maas-api"
Sep 06 06:20:57.497706 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497615    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee75e129-62bd-4069-932a-41d31ca90960" containerName="payload-processing"
Sep 06 06:20:57.497706 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497625    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee75e129-62bd-4069-932a-41d31ca90960" containerName="payload-processing"
Sep 06 06:20:57.497825 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497714    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cdbfe90-195f-4725-9be0-1b997d624edc" containerName="maas-api"
Sep 06 06:20:57.497825 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.497730    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee75e129-62bd-4069-932a-41d31ca90960" containerName="payload-processing"
Sep 06 06:20:57.501249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.501228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.503930 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.503912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ed85e8-serving-cert\""
Sep 06 06:20:57.513133 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.510343    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:20:57.520925 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.520896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:57.549295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.549258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzt89\" (UniqueName: \"kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.549475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.549323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.549475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.549376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.549475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.549423    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.621622 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.621581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:57.650646 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.650609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzt89\" (UniqueName: \"kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.650847 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.650673    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.650847 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.650696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.650847 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.650742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.650847 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:20:57.650840    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-serving-cert: secret "maas-api-e2e-api-a-ed85e8-serving-cert" not found
Sep 06 06:20:57.651087 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:20:57.650929    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls podName:a6adfeda-de69-4fa7-8099-1fb03dd7c58d nodeName:}" failed. No retries permitted until 2026-09-06 06:20:58.150909241 +0000 UTC m=+1611.854461962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls") pod "maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" (UID: "a6adfeda-de69-4fa7-8099-1fb03dd7c58d") : secret "maas-api-e2e-api-a-ed85e8-serving-cert" not found
Sep 06 06:20:57.651478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.651446    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.653477 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.653453    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.659833 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.659810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzt89\" (UniqueName: \"kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:57.872049 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:57.872011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:58.108171 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.108131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:58.155690 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.155603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:58.158164 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.158135    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") pod \"maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:58.203214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.203169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:20:58.248677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.248635    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:20:58.252417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.252397    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.255009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.254988    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ed85e8-dockercfg-wsbg2\""
Sep 06 06:20:58.255204 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.255187    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ed85e8\""
Sep 06 06:20:58.262728 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.262703    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:20:58.357504 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.357472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbv4z\" (UniqueName: \"kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.357680 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.357525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.421368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.421265    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:58.458493 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.458459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.458639 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.458546    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbv4z\" (UniqueName: \"kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.459437 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.459410    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.466835 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.466806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbv4z\" (UniqueName: \"kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z\") pod \"payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") " pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.565694 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.565657    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:58.697640 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.697607    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:20:58.698792 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:20:58.698765    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6adfeda_de69_4fa7_8099_1fb03dd7c58d.slice/crio-0ab23bef2b04b5757c03125fd8ce476cdc7d1fafdd100e25f131915415cf5758 WatchSource:0}: Error finding container 0ab23bef2b04b5757c03125fd8ce476cdc7d1fafdd100e25f131915415cf5758: Status 404 returned error can't find the container with id 0ab23bef2b04b5757c03125fd8ce476cdc7d1fafdd100e25f131915415cf5758
Sep 06 06:20:58.843483 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:58.843450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:20:58.844041 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:20:58.844016    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b37e90_8147_49b7_88df_73b6144db2e3.slice/crio-9bc819bf5664e8a1ba062286f696b20a439f49a48524dfc27b15a7219d6992d9 WatchSource:0}: Error finding container 9bc819bf5664e8a1ba062286f696b20a439f49a48524dfc27b15a7219d6992d9: Status 404 returned error can't find the container with id 9bc819bf5664e8a1ba062286f696b20a439f49a48524dfc27b15a7219d6992d9
Sep 06 06:20:59.332890 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.332850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" event={"ID":"a6adfeda-de69-4fa7-8099-1fb03dd7c58d","Type":"ContainerStarted","Data":"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"}
Sep 06 06:20:59.333141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.332919    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" event={"ID":"a6adfeda-de69-4fa7-8099-1fb03dd7c58d","Type":"ContainerStarted","Data":"0ab23bef2b04b5757c03125fd8ce476cdc7d1fafdd100e25f131915415cf5758"}
Sep 06 06:20:59.333141 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.332948    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:20:59.334750 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.334724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" event={"ID":"78b37e90-8147-49b7-88df-73b6144db2e3","Type":"ContainerStarted","Data":"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"}
Sep 06 06:20:59.334875 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.334773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" event={"ID":"78b37e90-8147-49b7-88df-73b6144db2e3","Type":"ContainerStarted","Data":"9bc819bf5664e8a1ba062286f696b20a439f49a48524dfc27b15a7219d6992d9"}
Sep 06 06:20:59.334875 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.334856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:20:59.351645 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.351573    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" podStartSLOduration=1.910735448 podStartE2EDuration="2.351552832s" totalImagesPullingTime="440.817384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:20:59.348550555 +0000 UTC m=+1613.052103303" watchObservedRunningTime="2026-09-06 06:20:59.351552832 +0000 UTC m=+1613.055105572"
Sep 06 06:20:59.365610 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:20:59.365558    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" podStartSLOduration=1.365544864 podStartE2EDuration="1.365544864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:20:59.363695217 +0000 UTC m=+1613.067247943" watchObservedRunningTime="2026-09-06 06:20:59.365544864 +0000 UTC m=+1613.069097651"
Sep 06 06:21:05.345657 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:05.345626    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:21:10.342024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:10.341987    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:21:14.027885 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:14.027850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:21.337585 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:21.337539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:21.819660 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:21.819629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:22.132420 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:22.132316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:22.307428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:22.307395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:25.571733 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:25.571690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:25.672171 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:25.672138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:26.720630 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:26.720597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:26.778978 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:26.778941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:26.879727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:26.879686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:27.194582 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.194551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:27.287823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.287697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:27.305158 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.305125    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:21:27.308512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.308489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.311053 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.311030    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1c55e7-dockercfg-v5gdm\""
Sep 06 06:21:27.311157 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.311091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1c55e7\""
Sep 06 06:21:27.318493 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.318466    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:21:27.437200 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.437150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.437393 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.437214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smx8m\" (UniqueName: \"kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.538030 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.537945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.538030 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.537992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smx8m\" (UniqueName: \"kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.538743 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.538718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.545906 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.545880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smx8m\" (UniqueName: \"kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m\") pod \"payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.620237 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.620200    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:27.892469 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:27.892440    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:21:27.894655 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:21:27.894630    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a42e1fc_75f7_40da_9c6a_1e036fd7cedf.slice/crio-d62fff7d03043808aad9f0759cfef9ab6d1a7e169e82ee974820a503099b9ff9 WatchSource:0}: Error finding container d62fff7d03043808aad9f0759cfef9ab6d1a7e169e82ee974820a503099b9ff9: Status 404 returned error can't find the container with id d62fff7d03043808aad9f0759cfef9ab6d1a7e169e82ee974820a503099b9ff9
Sep 06 06:21:28.500190 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:28.500154    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" event={"ID":"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf","Type":"ContainerStarted","Data":"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"}
Sep 06 06:21:28.500396 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:28.500220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" event={"ID":"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf","Type":"ContainerStarted","Data":"d62fff7d03043808aad9f0759cfef9ab6d1a7e169e82ee974820a503099b9ff9"}
Sep 06 06:21:28.500396 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:28.500248    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:28.517926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:28.517869    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" podStartSLOduration=1.5178527179999999 podStartE2EDuration="1.517852718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:21:28.515233771 +0000 UTC m=+1642.218786520" watchObservedRunningTime="2026-09-06 06:21:28.517852718 +0000 UTC m=+1642.221405453"
Sep 06 06:21:33.805072 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:33.805024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:35.718120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:35.718081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:39.507030 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:39.506996    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:21:47.601812 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:47.601767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:50.510433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:50.510397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:21:53.519928 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:21:53.519890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:02.722650 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:02.722607    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:03.404050 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:03.404012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:05.100443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:05.100414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.052130 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.052094    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:22:09.052977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.052941    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" podUID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" containerName="maas-api" containerID="cri-o://70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3" gracePeriod=30
Sep 06 06:22:09.081089 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.080741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.122866 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.122828    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:22:09.125095 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.123105    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" podUID="78b37e90-8147-49b7-88df-73b6144db2e3" containerName="payload-processing" containerID="cri-o://0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938" gracePeriod=30
Sep 06 06:22:09.179402 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.179366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.278511 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.278472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.332794 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.332721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.403507 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.403481    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:22:09.445069 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.445040    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:22:09.527536 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.527505    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzt89\" (UniqueName: \"kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89\") pod \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") "
Sep 06 06:22:09.527699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.527553    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") pod \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") "
Sep 06 06:22:09.527699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.527632    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca\") pod \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") "
Sep 06 06:22:09.527699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.527656    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs\") pod \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\" (UID: \"a6adfeda-de69-4fa7-8099-1fb03dd7c58d\") "
Sep 06 06:22:09.528051 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.528026    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6adfeda-de69-4fa7-8099-1fb03dd7c58d" (UID: "a6adfeda-de69-4fa7-8099-1fb03dd7c58d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:22:09.530016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.529983    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a6adfeda-de69-4fa7-8099-1fb03dd7c58d" (UID: "a6adfeda-de69-4fa7-8099-1fb03dd7c58d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:22:09.530150 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.530111    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6adfeda-de69-4fa7-8099-1fb03dd7c58d" (UID: "a6adfeda-de69-4fa7-8099-1fb03dd7c58d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:22:09.530232 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.530205    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89" (OuterVolumeSpecName: "kube-api-access-wzt89") pod "a6adfeda-de69-4fa7-8099-1fb03dd7c58d" (UID: "a6adfeda-de69-4fa7-8099-1fb03dd7c58d"). InnerVolumeSpecName "kube-api-access-wzt89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:22:09.614151 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.614055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:09.629040 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629013    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume\") pod \"78b37e90-8147-49b7-88df-73b6144db2e3\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") "
Sep 06 06:22:09.629225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629106    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbv4z\" (UniqueName: \"kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z\") pod \"78b37e90-8147-49b7-88df-73b6144db2e3\" (UID: \"78b37e90-8147-49b7-88df-73b6144db2e3\") "
Sep 06 06:22:09.629382 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629362    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzt89\" (UniqueName: \"kubernetes.io/projected/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-kube-api-access-wzt89\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.629455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629387    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.629455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629401    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.629455 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629415    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6adfeda-de69-4fa7-8099-1fb03dd7c58d-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.629580 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.629468    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78b37e90-8147-49b7-88df-73b6144db2e3" (UID: "78b37e90-8147-49b7-88df-73b6144db2e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:22:09.631276 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.631249    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z" (OuterVolumeSpecName: "kube-api-access-nbv4z") pod "78b37e90-8147-49b7-88df-73b6144db2e3" (UID: "78b37e90-8147-49b7-88df-73b6144db2e3"). InnerVolumeSpecName "kube-api-access-nbv4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:22:09.726995 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.726959    2567 generic.go:356] "Generic (PLEG): container finished" podID="78b37e90-8147-49b7-88df-73b6144db2e3" containerID="0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938" exitCode=0
Sep 06 06:22:09.727198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.727008    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"
Sep 06 06:22:09.727198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.727050    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" event={"ID":"78b37e90-8147-49b7-88df-73b6144db2e3","Type":"ContainerDied","Data":"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"}
Sep 06 06:22:09.727198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.727126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525" event={"ID":"78b37e90-8147-49b7-88df-73b6144db2e3","Type":"ContainerDied","Data":"9bc819bf5664e8a1ba062286f696b20a439f49a48524dfc27b15a7219d6992d9"}
Sep 06 06:22:09.727198 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.727155    2567 scope.go:117] "RemoveContainer" containerID="0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"
Sep 06 06:22:09.728527 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.728506    2567 generic.go:356] "Generic (PLEG): container finished" podID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" containerID="70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3" exitCode=0
Sep 06 06:22:09.728610 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.728563    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"
Sep 06 06:22:09.728610 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.728589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" event={"ID":"a6adfeda-de69-4fa7-8099-1fb03dd7c58d","Type":"ContainerDied","Data":"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"}
Sep 06 06:22:09.728699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.728630    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj" event={"ID":"a6adfeda-de69-4fa7-8099-1fb03dd7c58d","Type":"ContainerDied","Data":"0ab23bef2b04b5757c03125fd8ce476cdc7d1fafdd100e25f131915415cf5758"}
Sep 06 06:22:09.729934 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.729910    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbv4z\" (UniqueName: \"kubernetes.io/projected/78b37e90-8147-49b7-88df-73b6144db2e3-kube-api-access-nbv4z\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.730006 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.729939    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b37e90-8147-49b7-88df-73b6144db2e3-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:09.744806 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.744490    2567 scope.go:117] "RemoveContainer" containerID="0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"
Sep 06 06:22:09.744958 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:22:09.744923    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938\": container with ID starting with 0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938 not found: ID does not exist" containerID="0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"
Sep 06 06:22:09.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.745663    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938"} err="failed to get container status \"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938\": rpc error: code = NotFound desc = could not find container \"0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938\": container with ID starting with 0cf37bcccdeef36f52e52c17bea36d31b0a0b4cf6f73fdba4355184532ff2938 not found: ID does not exist"
Sep 06 06:22:09.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.745716    2567 scope.go:117] "RemoveContainer" containerID="70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"
Sep 06 06:22:09.756273 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.756254    2567 scope.go:117] "RemoveContainer" containerID="70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"
Sep 06 06:22:09.756572 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:22:09.756537    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3\": container with ID starting with 70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3 not found: ID does not exist" containerID="70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"
Sep 06 06:22:09.756572 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.756563    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3"} err="failed to get container status \"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3\": rpc error: code = NotFound desc = could not find container \"70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3\": container with ID starting with 70aecd7844b48059d0ff73d3de33284b17a35d6bd617edf101fd500e3f6fd2f3 not found: ID does not exist"
Sep 06 06:22:09.760848 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.760828    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:22:09.764182 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.764160    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed85e8-795c94684d-2gwhj"]
Sep 06 06:22:09.773758 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.773739    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:22:09.778255 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:09.778233    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ed85e8-d9f9bd449-pd525"]
Sep 06 06:22:10.787674 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:10.787638    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78b37e90-8147-49b7-88df-73b6144db2e3" path="/var/lib/kubelet/pods/78b37e90-8147-49b7-88df-73b6144db2e3/volumes"
Sep 06 06:22:10.788191 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:10.788175    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" path="/var/lib/kubelet/pods/a6adfeda-de69-4fa7-8099-1fb03dd7c58d/volumes"
Sep 06 06:22:15.753129 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:15.753081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:15.808570 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:15.808540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:16.059197 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:16.059159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:16.413381 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:16.413279    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:20.115446 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:20.115411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:21.136900 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:21.136865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:24.514902 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:24.514866    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:30.906225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906144    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906533    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" containerName="maas-api"
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906545    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" containerName="maas-api"
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906563    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78b37e90-8147-49b7-88df-73b6144db2e3" containerName="payload-processing"
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906569    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="78b37e90-8147-49b7-88df-73b6144db2e3" containerName="payload-processing"
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906632    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="78b37e90-8147-49b7-88df-73b6144db2e3" containerName="payload-processing"
Sep 06 06:22:30.906696 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.906642    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6adfeda-de69-4fa7-8099-1fb03dd7c58d" containerName="maas-api"
Sep 06 06:22:30.911095 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.911078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:30.914849 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.914826    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-429831\"/\"openshift-service-ca.crt\""
Sep 06 06:22:30.915092 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.914883    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-429831\"/\"kube-root-ca.crt\""
Sep 06 06:22:30.915187 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.914925    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-429831\"/\"default-dockercfg-2vjht\""
Sep 06 06:22:30.915262 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.914938    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-429831\"/\"rate-test-model-429831-kserve-self-signed-certs\""
Sep 06 06:22:30.918731 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.918708    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:22:30.933629 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:30.933598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:31.016354 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.016491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016380    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.016491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.016491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.016491 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016476    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.016635 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.016554    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkvp7\" (UniqueName: \"kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.117786 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.117786 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkvp7\" (UniqueName: \"kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.117939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118292 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.118182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118292 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.118267    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.118368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.118344    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.120200 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.120173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.120560 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.120542    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.126013 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.125990    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkvp7\" (UniqueName: \"kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7\") pod \"rate-test-model-429831-kserve-5fc95bf899-f6mdg\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") " pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.222345 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.222256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:31.494599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.494574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:22:31.496034 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:22:31.496007    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47bdea54_a9d1_436a_8713_07f55ce52d22.slice/crio-4bb4a0984b30c3d7e531a2f0a2d3631fd59a4c786800d4500f0901d6d79931d7 WatchSource:0}: Error finding container 4bb4a0984b30c3d7e531a2f0a2d3631fd59a4c786800d4500f0901d6d79931d7: Status 404 returned error can't find the container with id 4bb4a0984b30c3d7e531a2f0a2d3631fd59a4c786800d4500f0901d6d79931d7
Sep 06 06:22:31.734854 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.734821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:31.769548 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.769473    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:22:31.769745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.769718    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" podUID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" containerName="payload-pre-processing" containerID="cri-o://992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce" gracePeriod=30
Sep 06 06:22:31.834926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.834891    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:31.856909 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.856879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" event={"ID":"47bdea54-a9d1-436a-8713-07f55ce52d22","Type":"ContainerStarted","Data":"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"}
Sep 06 06:22:31.857045 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.856919    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" event={"ID":"47bdea54-a9d1-436a-8713-07f55ce52d22","Type":"ContainerStarted","Data":"4bb4a0984b30c3d7e531a2f0a2d3631fd59a4c786800d4500f0901d6d79931d7"}
Sep 06 06:22:31.873961 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.873910    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" podStartSLOduration=1.8738945870000001 podStartE2EDuration="1.873894587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:22:31.873370116 +0000 UTC m=+1705.576922855" watchObservedRunningTime="2026-09-06 06:22:31.873894587 +0000 UTC m=+1705.577447322"
Sep 06 06:22:31.922324 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.922278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:31.982498 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:31.982467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:32.014527 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.014494    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:22:32.127002 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.126969    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smx8m\" (UniqueName: \"kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m\") pod \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") "
Sep 06 06:22:32.127151 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.127096    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume\") pod \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\" (UID: \"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf\") "
Sep 06 06:22:32.127677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.127645    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" (UID: "3a42e1fc-75f7-40da-9c6a-1e036fd7cedf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:22:32.129441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.129418    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m" (OuterVolumeSpecName: "kube-api-access-smx8m") pod "3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" (UID: "3a42e1fc-75f7-40da-9c6a-1e036fd7cedf"). InnerVolumeSpecName "kube-api-access-smx8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:22:32.228639 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.228599    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smx8m\" (UniqueName: \"kubernetes.io/projected/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-kube-api-access-smx8m\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:32.228639 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.228641    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:32.266015 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.265985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:32.863104 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.863048    2567 generic.go:356] "Generic (PLEG): container finished" podID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" containerID="992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce" exitCode=0
Sep 06 06:22:32.863235 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.863101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" event={"ID":"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf","Type":"ContainerDied","Data":"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"}
Sep 06 06:22:32.863235 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.863155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl" event={"ID":"3a42e1fc-75f7-40da-9c6a-1e036fd7cedf","Type":"ContainerDied","Data":"d62fff7d03043808aad9f0759cfef9ab6d1a7e169e82ee974820a503099b9ff9"}
Sep 06 06:22:32.863235 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.863179    2567 scope.go:117] "RemoveContainer" containerID="992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"
Sep 06 06:22:32.863235 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.863107    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"
Sep 06 06:22:32.872688 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.872670    2567 scope.go:117] "RemoveContainer" containerID="992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"
Sep 06 06:22:32.872949 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:22:32.872931    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce\": container with ID starting with 992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce not found: ID does not exist" containerID="992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"
Sep 06 06:22:32.873012 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.872958    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce"} err="failed to get container status \"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce\": rpc error: code = NotFound desc = could not find container \"992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce\": container with ID starting with 992ff43efdc3f0e7f44f9ec89bef77ab527c406c51afc62eda87d78dd4fd17ce not found: ID does not exist"
Sep 06 06:22:32.880776 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.880752    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:22:32.889370 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:32.884521    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c55e7-57d978d4f8-pnmvl"]
Sep 06 06:22:34.787723 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:34.787691    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" path="/var/lib/kubelet/pods/3a42e1fc-75f7-40da-9c6a-1e036fd7cedf/volumes"
Sep 06 06:22:36.233451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:36.233416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:38.580777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:38.580734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:38.636869 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:38.636815    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:38.824882 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:38.824845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:39.241527 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:39.241492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:41.222472 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:41.222435    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:41.222951 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:41.222490    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:41.234977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:41.234954    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:41.927208 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:41.927172    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:22:49.584267 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.584228    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:22:49.584652 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.584427    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" podUID="10ca52c5-3600-4d91-8250-8757e74ad2b9" containerName="maas-api" containerID="cri-o://361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c" gracePeriod=30
Sep 06 06:22:49.608282 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.608248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:49.679771 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.679736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:49.740565 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.740470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:49.798205 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.798172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:49.841379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.841323    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:22:49.963143 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.963108    2567 generic.go:356] "Generic (PLEG): container finished" podID="10ca52c5-3600-4d91-8250-8757e74ad2b9" containerID="361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c" exitCode=0
Sep 06 06:22:49.963302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.963176    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"
Sep 06 06:22:49.963302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.963191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" event={"ID":"10ca52c5-3600-4d91-8250-8757e74ad2b9","Type":"ContainerDied","Data":"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"}
Sep 06 06:22:49.963302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.963258    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm" event={"ID":"10ca52c5-3600-4d91-8250-8757e74ad2b9","Type":"ContainerDied","Data":"7996249154a9828d8e7b4c752c8bfd6d181ea2d34419dcfddbacfeff806eb786"}
Sep 06 06:22:49.963302 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.963289    2567 scope.go:117] "RemoveContainer" containerID="361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"
Sep 06 06:22:49.974105 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.974052    2567 scope.go:117] "RemoveContainer" containerID="361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"
Sep 06 06:22:49.974430 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:22:49.974408    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c\": container with ID starting with 361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c not found: ID does not exist" containerID="361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"
Sep 06 06:22:49.974494 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.974441    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c"} err="failed to get container status \"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c\": rpc error: code = NotFound desc = could not find container \"361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c\": container with ID starting with 361fc635fa7f6e48e51a25635d8f4fae026b3c86605261572a5d424da4f4927c not found: ID does not exist"
Sep 06 06:22:49.991777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.991744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca\") pod \"10ca52c5-3600-4d91-8250-8757e74ad2b9\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") "
Sep 06 06:22:49.991908 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.991845    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lttbj\" (UniqueName: \"kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj\") pod \"10ca52c5-3600-4d91-8250-8757e74ad2b9\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") "
Sep 06 06:22:49.991908 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.991884    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") pod \"10ca52c5-3600-4d91-8250-8757e74ad2b9\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") "
Sep 06 06:22:49.992019 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.991946    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs\") pod \"10ca52c5-3600-4d91-8250-8757e74ad2b9\" (UID: \"10ca52c5-3600-4d91-8250-8757e74ad2b9\") "
Sep 06 06:22:49.992183 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.992157    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10ca52c5-3600-4d91-8250-8757e74ad2b9" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:22:49.992350 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.992330    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ca52c5-3600-4d91-8250-8757e74ad2b9-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:49.994173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.994148    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10ca52c5-3600-4d91-8250-8757e74ad2b9" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:22:49.994288 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.994268    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10ca52c5-3600-4d91-8250-8757e74ad2b9" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:22:49.994342 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:49.994321    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj" (OuterVolumeSpecName: "kube-api-access-lttbj") pod "10ca52c5-3600-4d91-8250-8757e74ad2b9" (UID: "10ca52c5-3600-4d91-8250-8757e74ad2b9"). InnerVolumeSpecName "kube-api-access-lttbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:22:50.093088 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.092987    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lttbj\" (UniqueName: \"kubernetes.io/projected/10ca52c5-3600-4d91-8250-8757e74ad2b9-kube-api-access-lttbj\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:50.093088 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.093015    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:50.093088 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.093024    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ca52c5-3600-4d91-8250-8757e74ad2b9-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:22:50.142197 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.142161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:50.287666 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.287635    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:22:50.289984 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.289960    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-340e00-f7b4d759b-xqjtm"]
Sep 06 06:22:50.786947 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:50.786912    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10ca52c5-3600-4d91-8250-8757e74ad2b9" path="/var/lib/kubelet/pods/10ca52c5-3600-4d91-8250-8757e74ad2b9/volumes"
Sep 06 06:22:55.738316 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:55.738282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:55.796210 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:55.796160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:55.985174 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:55.985136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:56.390264 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:56.390230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:22:58.633160 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:22:58.633122    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:06.703069 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.702996    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:23:06.704179 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.704129    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" podUID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" containerName="maas-api" containerID="cri-o://f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04" gracePeriod=30
Sep 06 06:23:06.727304 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.727267    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:06.800627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.800590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:06.858754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.858713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:06.920947 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.920912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:06.947370 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.947346    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:23:06.981171 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:06.981055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:07.054415 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.054389    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs\") pod \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") "
Sep 06 06:23:07.054587 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.054439    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca\") pod \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") "
Sep 06 06:23:07.054587 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.054463    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbldz\" (UniqueName: \"kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz\") pod \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") "
Sep 06 06:23:07.054587 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.054525    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") pod \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\" (UID: \"b2c43099-4f68-4d0d-bed8-c0650a77e6f5\") "
Sep 06 06:23:07.054856 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.054825    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2c43099-4f68-4d0d-bed8-c0650a77e6f5" (UID: "b2c43099-4f68-4d0d-bed8-c0650a77e6f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:23:07.056998 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.056969    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b2c43099-4f68-4d0d-bed8-c0650a77e6f5" (UID: "b2c43099-4f68-4d0d-bed8-c0650a77e6f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:23:07.056998 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.056989    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz" (OuterVolumeSpecName: "kube-api-access-mbldz") pod "b2c43099-4f68-4d0d-bed8-c0650a77e6f5" (UID: "b2c43099-4f68-4d0d-bed8-c0650a77e6f5"). InnerVolumeSpecName "kube-api-access-mbldz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:23:07.057215 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.056995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2c43099-4f68-4d0d-bed8-c0650a77e6f5" (UID: "b2c43099-4f68-4d0d-bed8-c0650a77e6f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:23:07.062881 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.062856    2567 generic.go:356] "Generic (PLEG): container finished" podID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" containerID="f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04" exitCode=0
Sep 06 06:23:07.062983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.062901    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"
Sep 06 06:23:07.062983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.062908    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" event={"ID":"b2c43099-4f68-4d0d-bed8-c0650a77e6f5","Type":"ContainerDied","Data":"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"}
Sep 06 06:23:07.062983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.062948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf" event={"ID":"b2c43099-4f68-4d0d-bed8-c0650a77e6f5","Type":"ContainerDied","Data":"10f1737f1352004ebe2469e5d746f4fe87ed32ed6a031e9e9728a90dcc162ec5"}
Sep 06 06:23:07.062983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.062968    2567 scope.go:117] "RemoveContainer" containerID="f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"
Sep 06 06:23:07.076801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.076720    2567 scope.go:117] "RemoveContainer" containerID="f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"
Sep 06 06:23:07.077125 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:23:07.077099    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04\": container with ID starting with f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04 not found: ID does not exist" containerID="f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"
Sep 06 06:23:07.077341 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.077294    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04"} err="failed to get container status \"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04\": rpc error: code = NotFound desc = could not find container \"f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04\": container with ID starting with f51f73be55c4a134f0d08e319bcde74b1e8a31a5814db293b1c036fb3e7b3d04 not found: ID does not exist"
Sep 06 06:23:07.089503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.089471    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:23:07.091427 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.091405    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b7c728-5d76c9cc8b-8snkf"]
Sep 06 06:23:07.156144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.156115    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:23:07.156144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.156140    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:23:07.156144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.156150    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbldz\" (UniqueName: \"kubernetes.io/projected/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-kube-api-access-mbldz\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:23:07.156144 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.156159    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2c43099-4f68-4d0d-bed8-c0650a77e6f5-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:23:07.253156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:07.253053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:08.786295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:08.786259    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" path="/var/lib/kubelet/pods/b2c43099-4f68-4d0d-bed8-c0650a77e6f5/volumes"
Sep 06 06:23:13.067459 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:13.067421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:13.126026 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:13.125984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:13.307377 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:13.307338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:20.211990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:20.211949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:46.497005 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:46.496964    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:23:46.515270 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:46.515235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:47.443184 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:47.443149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:47.855711 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:47.855664    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:48.140325 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:48.140225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:48.240411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:48.240369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:51.500375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:51.500333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:51.595201 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:51.595170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:52.809318 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:52.809276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:52.908939 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:52.908906    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:53.269611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.269520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:53.364859 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.364817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:53.414801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.414760    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:23:53.415206 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415188    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" containerName="payload-pre-processing"
Sep 06 06:23:53.415305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415208    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" containerName="payload-pre-processing"
Sep 06 06:23:53.415305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415223    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ca52c5-3600-4d91-8250-8757e74ad2b9" containerName="maas-api"
Sep 06 06:23:53.415305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415231    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ca52c5-3600-4d91-8250-8757e74ad2b9" containerName="maas-api"
Sep 06 06:23:53.415305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415266    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" containerName="maas-api"
Sep 06 06:23:53.415305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415274    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" containerName="maas-api"
Sep 06 06:23:53.415558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415366    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c43099-4f68-4d0d-bed8-c0650a77e6f5" containerName="maas-api"
Sep 06 06:23:53.415558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415381    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a42e1fc-75f7-40da-9c6a-1e036fd7cedf" containerName="payload-pre-processing"
Sep 06 06:23:53.415558 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.415391    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ca52c5-3600-4d91-8250-8757e74ad2b9" containerName="maas-api"
Sep 06 06:23:53.418617 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.418598    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.421036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.421016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a36847\""
Sep 06 06:23:53.421166 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.421023    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a36847-dockercfg-pmgsg\""
Sep 06 06:23:53.428924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.428899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:23:53.478858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.478806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmrpf\" (UniqueName: \"kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.479046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.478880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.579684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.579653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmrpf\" (UniqueName: \"kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.579865 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.579715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.580379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.580359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.587944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.587912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmrpf\" (UniqueName: \"kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf\") pod \"payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:53.713111 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.713078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:53.731028 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:53.730997    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:54.007003 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:54.006979    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:23:54.009007 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:23:54.008968    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cb9f09_c01d_4155_a2ae_fa72da8bfe6f.slice/crio-de96b61f38c85f599c882c421f1ac78ae2af127f70d7f55ac0f445ae3abc2c20 WatchSource:0}: Error finding container de96b61f38c85f599c882c421f1ac78ae2af127f70d7f55ac0f445ae3abc2c20: Status 404 returned error can't find the container with id de96b61f38c85f599c882c421f1ac78ae2af127f70d7f55ac0f445ae3abc2c20
Sep 06 06:23:54.313247 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:54.313211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" event={"ID":"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f","Type":"ContainerStarted","Data":"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"}
Sep 06 06:23:54.313453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:54.313264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" event={"ID":"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f","Type":"ContainerStarted","Data":"de96b61f38c85f599c882c421f1ac78ae2af127f70d7f55ac0f445ae3abc2c20"}
Sep 06 06:23:54.313453 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:54.313389    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:23:54.330802 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:54.330751    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" podStartSLOduration=1.330733481 podStartE2EDuration="1.330733481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:23:54.328441817 +0000 UTC m=+1788.031994553" watchObservedRunningTime="2026-09-06 06:23:54.330733481 +0000 UTC m=+1788.034286217"
Sep 06 06:23:58.755042 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:58.755007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:23:58.812395 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:23:58.812356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:01.518870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.518827    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" podUID="47bdea54-a9d1-436a-8713-07f55ce52d22" containerName="main" containerID="cri-o://3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077" gracePeriod=45
Sep 06 06:24:01.776691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.776623    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:24:01.856257 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856298    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856325    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkvp7\" (UniqueName: \"kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856367    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856390    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir\") pod \"47bdea54-a9d1-436a-8713-07f55ce52d22\" (UID: \"47bdea54-a9d1-436a-8713-07f55ce52d22\") "
Sep 06 06:24:01.856631 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856570    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache" (OuterVolumeSpecName: "model-cache") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:24:01.856704 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856677    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home" (OuterVolumeSpecName: "home") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:24:01.856798 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856784    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:01.856845 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856802    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:01.856845 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.856802    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:24:01.858566 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.858537    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm" (OuterVolumeSpecName: "dshm") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:24:01.858678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.858625    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:24:01.858678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.858637    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7" (OuterVolumeSpecName: "kube-api-access-nkvp7") pod "47bdea54-a9d1-436a-8713-07f55ce52d22" (UID: "47bdea54-a9d1-436a-8713-07f55ce52d22"). InnerVolumeSpecName "kube-api-access-nkvp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:24:01.957284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.957246    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:01.957284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.957278    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47bdea54-a9d1-436a-8713-07f55ce52d22-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:01.957284 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.957292    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkvp7\" (UniqueName: \"kubernetes.io/projected/47bdea54-a9d1-436a-8713-07f55ce52d22-kube-api-access-nkvp7\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:01.957526 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:01.957330    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47bdea54-a9d1-436a-8713-07f55ce52d22-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:24:02.359808 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.359768    2567 generic.go:356] "Generic (PLEG): container finished" podID="47bdea54-a9d1-436a-8713-07f55ce52d22" containerID="3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077" exitCode=0
Sep 06 06:24:02.359983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.359815    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"
Sep 06 06:24:02.359983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.359854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" event={"ID":"47bdea54-a9d1-436a-8713-07f55ce52d22","Type":"ContainerDied","Data":"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"}
Sep 06 06:24:02.359983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.359900    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg" event={"ID":"47bdea54-a9d1-436a-8713-07f55ce52d22","Type":"ContainerDied","Data":"4bb4a0984b30c3d7e531a2f0a2d3631fd59a4c786800d4500f0901d6d79931d7"}
Sep 06 06:24:02.359983 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.359921    2567 scope.go:117] "RemoveContainer" containerID="3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"
Sep 06 06:24:02.369255 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.369196    2567 scope.go:117] "RemoveContainer" containerID="3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"
Sep 06 06:24:02.369565 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:24:02.369454    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077\": container with ID starting with 3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077 not found: ID does not exist" containerID="3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"
Sep 06 06:24:02.369565 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.369495    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077"} err="failed to get container status \"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077\": rpc error: code = NotFound desc = could not find container \"3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077\": container with ID starting with 3325abbf426262eaaf4e00d1ffd722c263be71fc35df7d0f26c9d6c8bac2c077 not found: ID does not exist"
Sep 06 06:24:02.381829 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.381803    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:24:02.385278 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.385255    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-429831/rate-test-model-429831-kserve-5fc95bf899-f6mdg"]
Sep 06 06:24:02.786855 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:02.786776    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47bdea54-a9d1-436a-8713-07f55ce52d22" path="/var/lib/kubelet/pods/47bdea54-a9d1-436a-8713-07f55ce52d22/volumes"
Sep 06 06:24:05.321011 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:05.320922    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:24:16.358535 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:16.358490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:17.642310 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:17.642262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:17.889296 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:17.889259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:20.627322 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:20.627285    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:20.714090 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:20.714040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:21.769449 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:21.769417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:21.873037 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:21.872991    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:22.242287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.242255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:22.341882 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.341842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:22.384257 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.384225    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:24:22.384766 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.384742    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47bdea54-a9d1-436a-8713-07f55ce52d22" containerName="main"
Sep 06 06:24:22.384766 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.384765    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="47bdea54-a9d1-436a-8713-07f55ce52d22" containerName="main"
Sep 06 06:24:22.384963 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.384864    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="47bdea54-a9d1-436a-8713-07f55ce52d22" containerName="main"
Sep 06 06:24:22.387981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.387964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.396046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.396026    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a52b50\""
Sep 06 06:24:22.396319 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.396304    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a52b50-dockercfg-94rlr\""
Sep 06 06:24:22.404649 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.404624    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:24:22.441691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.441655    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:22.540646 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.540551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.540825 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.540643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd64n\" (UniqueName: \"kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.641750 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.641708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.641955 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.641759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd64n\" (UniqueName: \"kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.642628 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.642602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.654713 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.654680    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd64n\" (UniqueName: \"kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n\") pod \"payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.698421 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.698387    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:22.972971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:22.972940    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:24:22.974173 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:24:22.974147    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eefbc24_a9ec_4baa_9e64_e7544568de8c.slice/crio-42e2166afb6ec19417a58119584ee74b3e0d1553b6806fd2fb11ccd53038e055 WatchSource:0}: Error finding container 42e2166afb6ec19417a58119584ee74b3e0d1553b6806fd2fb11ccd53038e055: Status 404 returned error can't find the container with id 42e2166afb6ec19417a58119584ee74b3e0d1553b6806fd2fb11ccd53038e055
Sep 06 06:24:23.476478 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:23.476439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" event={"ID":"9eefbc24-a9ec-4baa-9e64-e7544568de8c","Type":"ContainerStarted","Data":"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"}
Sep 06 06:24:23.476660 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:23.476495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" event={"ID":"9eefbc24-a9ec-4baa-9e64-e7544568de8c","Type":"ContainerStarted","Data":"42e2166afb6ec19417a58119584ee74b3e0d1553b6806fd2fb11ccd53038e055"}
Sep 06 06:24:23.476660 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:23.476532    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:23.495174 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:23.495121    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" podStartSLOduration=1.495105213 podStartE2EDuration="1.495105213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:24:23.492108902 +0000 UTC m=+1817.195661639" watchObservedRunningTime="2026-09-06 06:24:23.495105213 +0000 UTC m=+1817.198657948"
Sep 06 06:24:34.488185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:34.488153    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:24:45.575676 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:45.575639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:24:50.705732 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:50.705698    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs_9eefbc24-a9ec-4baa-9e64-e7544568de8c/payload-processing/0.log"
Sep 06 06:24:56.018185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:24:56.018148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:23.170977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:23.170945    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9_65cb9f09-c01d-4155-a2ae-fa72da8bfe6f/payload-processing/0.log"
Sep 06 06:25:23.392053 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:23.392022    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs_9eefbc24-a9ec-4baa-9e64-e7544568de8c/payload-processing/0.log"
Sep 06 06:25:23.648248 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:23.648213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:23.937103 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:23.936999    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:24.468600 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:24.468553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:27.015990 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:27.015946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:27.106697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:27.106660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:28.173761 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:28.173718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:28.274295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:28.274243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:28.327510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:28.327470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:28.723986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:28.723951    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:28.824484 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:28.824444    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:51.950500 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:51.950466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:58.656431 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:58.656390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:58.724619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:58.724572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:58.788525 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:58.788440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:58.844740 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:58.844699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:58.893830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:58.893791    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:25:59.191175 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:25:59.191131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:04.954216 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:04.954174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:05.008713 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:05.008681    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:05.284515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:05.284425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:08.979420 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:08.979377    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:24.243214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.243160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:24.273708 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.273669    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:26:24.273986 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.273958    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" podUID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" containerName="payload-processing" containerID="cri-o://f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17" gracePeriod=30
Sep 06 06:26:24.342436 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.342398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:24.434147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.434114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:24.491476 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.491443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:24.522290 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.522266    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:26:24.656009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.655974    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmrpf\" (UniqueName: \"kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf\") pod \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") "
Sep 06 06:26:24.656009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.656013    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume\") pod \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\" (UID: \"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f\") "
Sep 06 06:26:24.656407 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.656384    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" (UID: "65cb9f09-c01d-4155-a2ae-fa72da8bfe6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:26:24.658167 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.658141    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf" (OuterVolumeSpecName: "kube-api-access-xmrpf") pod "65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" (UID: "65cb9f09-c01d-4155-a2ae-fa72da8bfe6f"). InnerVolumeSpecName "kube-api-access-xmrpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:26:24.757260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.757177    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:26:24.757260 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.757207    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmrpf\" (UniqueName: \"kubernetes.io/projected/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f-kube-api-access-xmrpf\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:26:24.799653 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:24.799602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:25.140035 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.139998    2567 generic.go:356] "Generic (PLEG): container finished" podID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" containerID="f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17" exitCode=0
Sep 06 06:26:25.140225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.140040    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"
Sep 06 06:26:25.140225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.140089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" event={"ID":"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f","Type":"ContainerDied","Data":"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"}
Sep 06 06:26:25.140225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.140143    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9" event={"ID":"65cb9f09-c01d-4155-a2ae-fa72da8bfe6f","Type":"ContainerDied","Data":"de96b61f38c85f599c882c421f1ac78ae2af127f70d7f55ac0f445ae3abc2c20"}
Sep 06 06:26:25.140225 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.140169    2567 scope.go:117] "RemoveContainer" containerID="f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"
Sep 06 06:26:25.149097 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.148983    2567 scope.go:117] "RemoveContainer" containerID="f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"
Sep 06 06:26:25.149409 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:26:25.149387    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17\": container with ID starting with f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17 not found: ID does not exist" containerID="f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"
Sep 06 06:26:25.149518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.149421    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17"} err="failed to get container status \"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17\": rpc error: code = NotFound desc = could not find container \"f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17\": container with ID starting with f63080a9bedd98c5de3d6f06d335faeef8c2e416b4df5df6479de09871426b17 not found: ID does not exist"
Sep 06 06:26:25.158432 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.158409    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:26:25.162681 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:25.162658    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a36847-8447cdf59-88cv9"]
Sep 06 06:26:26.791233 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:26.789527    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" path="/var/lib/kubelet/pods/65cb9f09-c01d-4155-a2ae-fa72da8bfe6f/volumes"
Sep 06 06:26:29.865336 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:29.865301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:34.438449 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:34.438410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:40.591239 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:40.591201    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:26:40.646436 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:26:40.646395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:03.519737 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:03.519648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:06.547836 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:06.547800    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:06.813251 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:06.813225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:12.284911 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:12.284873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:26.422863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:26.422819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:26.521625 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:26.521586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:26.685660 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:26.685552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:27.178518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:27.178480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:37.525008 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.524915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:37.551458 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.551427    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:27:37.551655 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.551628    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" podUID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" containerName="payload-processing" containerID="cri-o://3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4" gracePeriod=30
Sep 06 06:27:37.626375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.626343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:37.717310 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.717022    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:37.781036 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.780992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:37.796440 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.796417    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:27:37.823207 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.823170    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume\") pod \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") "
Sep 06 06:27:37.823349 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.823285    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd64n\" (UniqueName: \"kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n\") pod \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\" (UID: \"9eefbc24-a9ec-4baa-9e64-e7544568de8c\") "
Sep 06 06:27:37.823595 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.823565    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9eefbc24-a9ec-4baa-9e64-e7544568de8c" (UID: "9eefbc24-a9ec-4baa-9e64-e7544568de8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:27:37.825483 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.825461    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n" (OuterVolumeSpecName: "kube-api-access-vd64n") pod "9eefbc24-a9ec-4baa-9e64-e7544568de8c" (UID: "9eefbc24-a9ec-4baa-9e64-e7544568de8c"). InnerVolumeSpecName "kube-api-access-vd64n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:27:37.924962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.924925    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eefbc24-a9ec-4baa-9e64-e7544568de8c-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:37.924962 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:37.924959    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd64n\" (UniqueName: \"kubernetes.io/projected/9eefbc24-a9ec-4baa-9e64-e7544568de8c-kube-api-access-vd64n\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:38.098273 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.098233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:38.533599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.533489    2567 generic.go:356] "Generic (PLEG): container finished" podID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" containerID="3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4" exitCode=0
Sep 06 06:27:38.533599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.533553    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"
Sep 06 06:27:38.533599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.533579    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" event={"ID":"9eefbc24-a9ec-4baa-9e64-e7544568de8c","Type":"ContainerDied","Data":"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"}
Sep 06 06:27:38.534214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.533684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs" event={"ID":"9eefbc24-a9ec-4baa-9e64-e7544568de8c","Type":"ContainerDied","Data":"42e2166afb6ec19417a58119584ee74b3e0d1553b6806fd2fb11ccd53038e055"}
Sep 06 06:27:38.534214 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.533716    2567 scope.go:117] "RemoveContainer" containerID="3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"
Sep 06 06:27:38.544114 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.544098    2567 scope.go:117] "RemoveContainer" containerID="3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"
Sep 06 06:27:38.544373 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:27:38.544356    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4\": container with ID starting with 3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4 not found: ID does not exist" containerID="3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"
Sep 06 06:27:38.544427 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.544380    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4"} err="failed to get container status \"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4\": rpc error: code = NotFound desc = could not find container \"3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4\": container with ID starting with 3656e3fe9cd126d897934ced9507b5177f668332a63a55ba90c4eacd16d868a4 not found: ID does not exist"
Sep 06 06:27:38.555893 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.555871    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:27:38.562424 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.562402    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a52b50-65788bfd85-w2brs"]
Sep 06 06:27:38.786510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:38.786424    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" path="/var/lib/kubelet/pods/9eefbc24-a9ec-4baa-9e64-e7544568de8c/volumes"
Sep 06 06:27:43.850213 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:43.850180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:43.950416 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:43.950378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:44.118046 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:44.117962    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:44.573647 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:44.573610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:54.903977 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.903939    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:27:54.904468 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.904222    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" podUID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" containerName="maas-api" containerID="cri-o://b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694" gracePeriod=30
Sep 06 06:27:54.928606 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.928544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:54.954117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.954044    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:27:54.954289 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.954262    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" podUID="58a6afbc-7263-4294-8cb8-62a42bc405c1" containerName="payload-processing" containerID="cri-o://bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526" gracePeriod=30
Sep 06 06:27:54.961502 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.961343    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:27:54.961642 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.961550    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" podUID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" containerName="payload-pre-processing" containerID="cri-o://1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7" gracePeriod=30
Sep 06 06:27:54.993661 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:54.993625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:55.093470 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.093427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:55.158957 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.158689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:55.239832 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.239809    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:27:55.244619 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.244602    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:27:55.271747 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.271726    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:27:55.375576 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375539    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4lwj\" (UniqueName: \"kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj\") pod \"58a6afbc-7263-4294-8cb8-62a42bc405c1\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") "
Sep 06 06:27:55.375767 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375580    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume\") pod \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") "
Sep 06 06:27:55.375767 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375614    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") pod \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") "
Sep 06 06:27:55.375767 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375738    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh2km\" (UniqueName: \"kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km\") pod \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") "
Sep 06 06:27:55.375950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375780    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs\") pod \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") "
Sep 06 06:27:55.375950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375835    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca\") pod \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\" (UID: \"2761ca3c-378c-4c78-b2c7-692ac3b01e6e\") "
Sep 06 06:27:55.375950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375873    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume\") pod \"58a6afbc-7263-4294-8cb8-62a42bc405c1\" (UID: \"58a6afbc-7263-4294-8cb8-62a42bc405c1\") "
Sep 06 06:27:55.375950 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.375947    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc6qf\" (UniqueName: \"kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf\") pod \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\" (UID: \"e66728df-2ebd-48e8-8c32-607dfc52fcbb\") "
Sep 06 06:27:55.376169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.376019    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e66728df-2ebd-48e8-8c32-607dfc52fcbb" (UID: "e66728df-2ebd-48e8-8c32-607dfc52fcbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:27:55.376409 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.376384    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e66728df-2ebd-48e8-8c32-607dfc52fcbb-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.376508 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.376434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58a6afbc-7263-4294-8cb8-62a42bc405c1" (UID: "58a6afbc-7263-4294-8cb8-62a42bc405c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:27:55.376644 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.376606    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2761ca3c-378c-4c78-b2c7-692ac3b01e6e" (UID: "2761ca3c-378c-4c78-b2c7-692ac3b01e6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:27:55.378016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.377984    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2761ca3c-378c-4c78-b2c7-692ac3b01e6e" (UID: "2761ca3c-378c-4c78-b2c7-692ac3b01e6e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:27:55.378016 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.377990    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km" (OuterVolumeSpecName: "kube-api-access-wh2km") pod "2761ca3c-378c-4c78-b2c7-692ac3b01e6e" (UID: "2761ca3c-378c-4c78-b2c7-692ac3b01e6e"). InnerVolumeSpecName "kube-api-access-wh2km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:27:55.378202 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.378033    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf" (OuterVolumeSpecName: "kube-api-access-rc6qf") pod "e66728df-2ebd-48e8-8c32-607dfc52fcbb" (UID: "e66728df-2ebd-48e8-8c32-607dfc52fcbb"). InnerVolumeSpecName "kube-api-access-rc6qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:27:55.378257 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.378244    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2761ca3c-378c-4c78-b2c7-692ac3b01e6e" (UID: "2761ca3c-378c-4c78-b2c7-692ac3b01e6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:27:55.378295 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.378251    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj" (OuterVolumeSpecName: "kube-api-access-k4lwj") pod "58a6afbc-7263-4294-8cb8-62a42bc405c1" (UID: "58a6afbc-7263-4294-8cb8-62a42bc405c1"). InnerVolumeSpecName "kube-api-access-k4lwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:27:55.463314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.463224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:27:55.477281 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477256    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc6qf\" (UniqueName: \"kubernetes.io/projected/e66728df-2ebd-48e8-8c32-607dfc52fcbb-kube-api-access-rc6qf\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477281 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477281    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4lwj\" (UniqueName: \"kubernetes.io/projected/58a6afbc-7263-4294-8cb8-62a42bc405c1-kube-api-access-k4lwj\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477291    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477301    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh2km\" (UniqueName: \"kubernetes.io/projected/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-kube-api-access-wh2km\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477311    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477320    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2761ca3c-378c-4c78-b2c7-692ac3b01e6e-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.477411 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.477328    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a6afbc-7263-4294-8cb8-62a42bc405c1-plugins-config-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:27:55.630756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.630722    2567 generic.go:356] "Generic (PLEG): container finished" podID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" containerID="b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694" exitCode=0
Sep 06 06:27:55.630924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.630765    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"
Sep 06 06:27:55.630924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.630807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" event={"ID":"2761ca3c-378c-4c78-b2c7-692ac3b01e6e","Type":"ContainerDied","Data":"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"}
Sep 06 06:27:55.630924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.630861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4" event={"ID":"2761ca3c-378c-4c78-b2c7-692ac3b01e6e","Type":"ContainerDied","Data":"7ca23479943f61e78c121982bd64cc5ae8e6f4cc0c190c35258c147e0315a886"}
Sep 06 06:27:55.630924 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.630889    2567 scope.go:117] "RemoveContainer" containerID="b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"
Sep 06 06:27:55.632430 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.632401    2567 generic.go:356] "Generic (PLEG): container finished" podID="58a6afbc-7263-4294-8cb8-62a42bc405c1" containerID="bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526" exitCode=0
Sep 06 06:27:55.632551 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.632443    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"
Sep 06 06:27:55.632551 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.632491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" event={"ID":"58a6afbc-7263-4294-8cb8-62a42bc405c1","Type":"ContainerDied","Data":"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"}
Sep 06 06:27:55.632672 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.632557    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r" event={"ID":"58a6afbc-7263-4294-8cb8-62a42bc405c1","Type":"ContainerDied","Data":"9d6caa2868fea957ac1d72ca5467901d5de03771fae92fc0e1d0079af7fa6f94"}
Sep 06 06:27:55.634355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.634221    2567 generic.go:356] "Generic (PLEG): container finished" podID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" containerID="1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7" exitCode=0
Sep 06 06:27:55.634355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.634257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" event={"ID":"e66728df-2ebd-48e8-8c32-607dfc52fcbb","Type":"ContainerDied","Data":"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"}
Sep 06 06:27:55.634355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.634265    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"
Sep 06 06:27:55.634355 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.634295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb" event={"ID":"e66728df-2ebd-48e8-8c32-607dfc52fcbb","Type":"ContainerDied","Data":"06c05544d3d99a2f07e160240ea05bd3e7b1f3b7d5b471b757404918d44c3afc"}
Sep 06 06:27:55.642272 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.642255    2567 scope.go:117] "RemoveContainer" containerID="b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"
Sep 06 06:27:55.642581 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:27:55.642533    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694\": container with ID starting with b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694 not found: ID does not exist" containerID="b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"
Sep 06 06:27:55.642677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.642586    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694"} err="failed to get container status \"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694\": rpc error: code = NotFound desc = could not find container \"b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694\": container with ID starting with b1fbd2ff0ff3d69a1105ee652c0ede100dc8d77b52c849f5375f4c996487e694 not found: ID does not exist"
Sep 06 06:27:55.642677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.642601    2567 scope.go:117] "RemoveContainer" containerID="bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"
Sep 06 06:27:55.651475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.651462    2567 scope.go:117] "RemoveContainer" containerID="bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"
Sep 06 06:27:55.651711 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:27:55.651691    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526\": container with ID starting with bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526 not found: ID does not exist" containerID="bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"
Sep 06 06:27:55.651756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.651718    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526"} err="failed to get container status \"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526\": rpc error: code = NotFound desc = could not find container \"bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526\": container with ID starting with bedb281125c8b7b27a55b08a6a3faee48aa08cc340e689e9d3ccdd191c866526 not found: ID does not exist"
Sep 06 06:27:55.651756 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.651733    2567 scope.go:117] "RemoveContainer" containerID="1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"
Sep 06 06:27:55.655805 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.655782    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:27:55.659127 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.659106    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-429831-7dccb9d4f8-4vbr4"]
Sep 06 06:27:55.660583 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.660569    2567 scope.go:117] "RemoveContainer" containerID="1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"
Sep 06 06:27:55.660844 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:27:55.660822    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7\": container with ID starting with 1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7 not found: ID does not exist" containerID="1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"
Sep 06 06:27:55.660915 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.660851    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7"} err="failed to get container status \"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7\": rpc error: code = NotFound desc = could not find container \"1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7\": container with ID starting with 1d124c2bc51e87200e2b7971b31eae1cf369a467bb4180896dfa8e5d59b38ca7 not found: ID does not exist"
Sep 06 06:27:55.669464 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.669431    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:27:55.671569 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.671544    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-429831-b7f6798cd-9f8hb"]
Sep 06 06:27:55.681952 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.681928    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:27:55.693207 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:55.693179    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-429831-7869d5588f-pfz9r"]
Sep 06 06:27:56.786415 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:56.786383    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" path="/var/lib/kubelet/pods/2761ca3c-378c-4c78-b2c7-692ac3b01e6e/volumes"
Sep 06 06:27:56.786768 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:56.786731    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a6afbc-7263-4294-8cb8-62a42bc405c1" path="/var/lib/kubelet/pods/58a6afbc-7263-4294-8cb8-62a42bc405c1/volumes"
Sep 06 06:27:56.787043 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:27:56.787031    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" path="/var/lib/kubelet/pods/e66728df-2ebd-48e8-8c32-607dfc52fcbb/volumes"
Sep 06 06:28:01.399103 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:01.399070    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:01.499659 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:01.499617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:01.638389 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:01.638353    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:28:01.638641 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:01.638606    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" containerID="cri-o://491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87" gracePeriod=30
Sep 06 06:28:01.655673 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:01.655596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:02.196124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:02.196086    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:04.779130 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:04.779093    2567 patch_prober.go:28] interesting pod/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 06:28:04.779515 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:04.779162    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 06:28:06.882773 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.882750    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:28:06.980814 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980703    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6n47\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.980814 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980767    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.980814 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980801    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980913    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980964    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.980990    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981030    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981110    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981155 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981138    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo\") pod \"721aa5f0-1d56-4a34-aef7-534353ae35b3\" (UID: \"721aa5f0-1d56-4a34-aef7-534353ae35b3\") "
Sep 06 06:28:06.981503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981157    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:28:06.981503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981244    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:28:06.981503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981348    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:28:06.981503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981361    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:28:06.981503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:28:06.981754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981535    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721aa5f0-1d56-4a34-aef7-534353ae35b3-istiod-ca-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:06.981754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981555    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-data\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:06.981754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981564    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-credential-socket\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:06.981754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981573    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-socket\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:06.981754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.981581    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-workload-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:06.983269 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.983241    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 06:28:06.983354 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.983324    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:28:06.983599 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.983571    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47" (OuterVolumeSpecName: "kube-api-access-z6n47") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "kube-api-access-z6n47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:28:06.983692 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:06.983594    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "721aa5f0-1d56-4a34-aef7-534353ae35b3" (UID: "721aa5f0-1d56-4a34-aef7-534353ae35b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:28:07.082574 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.082531    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-token\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:07.082574 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.082566    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-envoy\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:07.082574 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.082576    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721aa5f0-1d56-4a34-aef7-534353ae35b3-istio-podinfo\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:07.082574 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.082584    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6n47\" (UniqueName: \"kubernetes.io/projected/721aa5f0-1d56-4a34-aef7-534353ae35b3-kube-api-access-z6n47\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:07.702933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.702892    2567 generic.go:356] "Generic (PLEG): container finished" podID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerID="491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87" exitCode=0
Sep 06 06:28:07.703147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.702956    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"
Sep 06 06:28:07.703147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.702987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerDied","Data":"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"}
Sep 06 06:28:07.703147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.703076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr" event={"ID":"721aa5f0-1d56-4a34-aef7-534353ae35b3","Type":"ContainerDied","Data":"5f2c18c770a2eb7423e79ec56a9d6d68d18ff2ca4e30652a63b562d22ce9cfbd"}
Sep 06 06:28:07.703147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.703110    2567 scope.go:117] "RemoveContainer" containerID="491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"
Sep 06 06:28:07.713006 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.712982    2567 scope.go:117] "RemoveContainer" containerID="ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"
Sep 06 06:28:07.721162 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.721141    2567 scope.go:117] "RemoveContainer" containerID="491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"
Sep 06 06:28:07.721422 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:28:07.721402    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87\": container with ID starting with 491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87 not found: ID does not exist" containerID="491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"
Sep 06 06:28:07.721476 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.721431    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87"} err="failed to get container status \"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87\": rpc error: code = NotFound desc = could not find container \"491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87\": container with ID starting with 491f14e00dba2bdaddc39b54b469c405028a997895eb8ee1bca398af3247ff87 not found: ID does not exist"
Sep 06 06:28:07.721476 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.721450    2567 scope.go:117] "RemoveContainer" containerID="ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"
Sep 06 06:28:07.721729 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:28:07.721711    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53\": container with ID starting with ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53 not found: ID does not exist" containerID="ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"
Sep 06 06:28:07.721796 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.721734    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53"} err="failed to get container status \"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53\": rpc error: code = NotFound desc = could not find container \"ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53\": container with ID starting with ce58d40aaffec739682c4f44d33d2efd6ffe8f1fd72e8c4b0a606fb20311bf53 not found: ID does not exist"
Sep 06 06:28:07.731602 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.729472    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:28:07.735031 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:07.735010    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-429831-openshift-default-fd5b96d54-7jfzr"]
Sep 06 06:28:08.786566 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:08.786534    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" path="/var/lib/kubelet/pods/721aa5f0-1d56-4a34-aef7-534353ae35b3/volumes"
Sep 06 06:28:12.401908 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.401869    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:28:12.402413 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.402126    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" podUID="b7346d10-91b5-4170-b050-43ff825e95a5" containerName="maas-api" containerID="cri-o://4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f" gracePeriod=30
Sep 06 06:28:12.426906 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.426876    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:12.491832 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.491795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:12.549462 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.549426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:12.606443 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.606403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:12.647172 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.647148    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:28:12.729156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.729046    2567 generic.go:356] "Generic (PLEG): container finished" podID="b7346d10-91b5-4170-b050-43ff825e95a5" containerID="4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f" exitCode=0
Sep 06 06:28:12.729156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.729106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" event={"ID":"b7346d10-91b5-4170-b050-43ff825e95a5","Type":"ContainerDied","Data":"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"}
Sep 06 06:28:12.729156 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.729132    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"
Sep 06 06:28:12.729383 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.729194    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785" event={"ID":"b7346d10-91b5-4170-b050-43ff825e95a5","Type":"ContainerDied","Data":"fcd3fe64c95072d4a0aebb0a12002c7324598de9820af873e703be760905f84c"}
Sep 06 06:28:12.729383 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.729219    2567 scope.go:117] "RemoveContainer" containerID="4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"
Sep 06 06:28:12.732539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732510    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca\") pod \"b7346d10-91b5-4170-b050-43ff825e95a5\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") "
Sep 06 06:28:12.732677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732575    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") pod \"b7346d10-91b5-4170-b050-43ff825e95a5\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") "
Sep 06 06:28:12.732677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732627    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4wcg\" (UniqueName: \"kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg\") pod \"b7346d10-91b5-4170-b050-43ff825e95a5\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") "
Sep 06 06:28:12.732677 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732654    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs\") pod \"b7346d10-91b5-4170-b050-43ff825e95a5\" (UID: \"b7346d10-91b5-4170-b050-43ff825e95a5\") "
Sep 06 06:28:12.732850 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732817    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7346d10-91b5-4170-b050-43ff825e95a5" (UID: "b7346d10-91b5-4170-b050-43ff825e95a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:28:12.732939 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.732917    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7346d10-91b5-4170-b050-43ff825e95a5-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:12.735194 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.735168    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b7346d10-91b5-4170-b050-43ff825e95a5" (UID: "b7346d10-91b5-4170-b050-43ff825e95a5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:28:12.735512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.735343    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7346d10-91b5-4170-b050-43ff825e95a5" (UID: "b7346d10-91b5-4170-b050-43ff825e95a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:28:12.735512 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.735352    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg" (OuterVolumeSpecName: "kube-api-access-q4wcg") pod "b7346d10-91b5-4170-b050-43ff825e95a5" (UID: "b7346d10-91b5-4170-b050-43ff825e95a5"). InnerVolumeSpecName "kube-api-access-q4wcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:28:12.739524 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.739507    2567 scope.go:117] "RemoveContainer" containerID="4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"
Sep 06 06:28:12.739840 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:28:12.739820    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f\": container with ID starting with 4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f not found: ID does not exist" containerID="4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"
Sep 06 06:28:12.739917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.739848    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f"} err="failed to get container status \"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f\": rpc error: code = NotFound desc = could not find container \"4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f\": container with ID starting with 4811da8c25a3651a5b4dbeb72b7f40eddbbbce078ddbd0651e86fe095877eb5f not found: ID does not exist"
Sep 06 06:28:12.834195 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.834163    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4wcg\" (UniqueName: \"kubernetes.io/projected/b7346d10-91b5-4170-b050-43ff825e95a5-kube-api-access-q4wcg\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:12.834195 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.834192    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:12.834195 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.834203    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7346d10-91b5-4170-b050-43ff825e95a5-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:12.966251 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:12.966204    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:13.045718 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:13.045640    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:28:13.050180 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:13.050154    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-dc97f9-5b547977dd-g8785"]
Sep 06 06:28:14.786474 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:14.786442    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7346d10-91b5-4170-b050-43ff825e95a5" path="/var/lib/kubelet/pods/b7346d10-91b5-4170-b050-43ff825e95a5/volumes"
Sep 06 06:28:18.723044 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:18.722972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:18.793530 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:18.793491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:18.978147 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:18.978042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:21.021075 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.021027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:28:21.543681 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.543643    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:28:21.543938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.543908    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55947c9784-n24nx" podUID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" containerName="manager" containerID="cri-o://983cd9a58ab1bcf8da4e36509e40f531e8c574fe7030bcfb78fc2399fff5abd1" gracePeriod=30
Sep 06 06:28:21.780098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.780054    2567 generic.go:356] "Generic (PLEG): container finished" podID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" containerID="983cd9a58ab1bcf8da4e36509e40f531e8c574fe7030bcfb78fc2399fff5abd1" exitCode=0
Sep 06 06:28:21.780098 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.780090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-n24nx" event={"ID":"3bc95d52-1024-48e8-934b-fa4c4d1734d7","Type":"ContainerDied","Data":"983cd9a58ab1bcf8da4e36509e40f531e8c574fe7030bcfb78fc2399fff5abd1"}
Sep 06 06:28:21.780294 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.780140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-n24nx" event={"ID":"3bc95d52-1024-48e8-934b-fa4c4d1734d7","Type":"ContainerDied","Data":"f5b2faecfe0209240f1d05a34df2cb73d3d49ccfd83aa94f963f6427ee869151"}
Sep 06 06:28:21.780294 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.780160    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5b2faecfe0209240f1d05a34df2cb73d3d49ccfd83aa94f963f6427ee869151"
Sep 06 06:28:21.791018 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.790998    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:28:21.808540 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.808439    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs\") pod \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") "
Sep 06 06:28:21.808540 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.808482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr7rd\" (UniqueName: \"kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd\") pod \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") "
Sep 06 06:28:21.809458 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.808568    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert\") pod \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\" (UID: \"3bc95d52-1024-48e8-934b-fa4c4d1734d7\") "
Sep 06 06:28:21.810843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.810817    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3bc95d52-1024-48e8-934b-fa4c4d1734d7" (UID: "3bc95d52-1024-48e8-934b-fa4c4d1734d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:28:21.811166 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.811137    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3bc95d52-1024-48e8-934b-fa4c4d1734d7" (UID: "3bc95d52-1024-48e8-934b-fa4c4d1734d7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:28:21.811593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.811571    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd" (OuterVolumeSpecName: "kube-api-access-rr7rd") pod "3bc95d52-1024-48e8-934b-fa4c4d1734d7" (UID: "3bc95d52-1024-48e8-934b-fa4c4d1734d7"). InnerVolumeSpecName "kube-api-access-rr7rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:28:21.909120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.909086    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:21.909120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.909107    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr7rd\" (UniqueName: \"kubernetes.io/projected/3bc95d52-1024-48e8-934b-fa4c4d1734d7-kube-api-access-rr7rd\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:21.909120 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:21.909116    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc95d52-1024-48e8-934b-fa4c4d1734d7-webhook-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:28:22.788454 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:22.788426    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-n24nx"
Sep 06 06:28:22.812686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:22.812645    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:28:22.816117 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:22.816094    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55947c9784-n24nx"]
Sep 06 06:28:24.785708 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:28:24.785670    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" path="/var/lib/kubelet/pods/3bc95d52-1024-48e8-934b-fa4c4d1734d7/volumes"
Sep 06 06:29:07.946601 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:07.946565    2567 scope.go:117] "RemoveContainer" containerID="983cd9a58ab1bcf8da4e36509e40f531e8c574fe7030bcfb78fc2399fff5abd1"
Sep 06 06:29:21.656859 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:21.656815    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:29:23.597441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597408    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597784    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597804    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597816    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" containerName="maas-api"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597826    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" containerName="maas-api"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597839    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7346d10-91b5-4170-b050-43ff825e95a5" containerName="maas-api"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597848    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7346d10-91b5-4170-b050-43ff825e95a5" containerName="maas-api"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597860    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" containerName="manager"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597865    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" containerName="manager"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597874    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" containerName="payload-pre-processing"
Sep 06 06:29:23.597870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597879    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" containerName="payload-pre-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597886    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597892    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597909    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a6afbc-7263-4294-8cb8-62a42bc405c1" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597918    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a6afbc-7263-4294-8cb8-62a42bc405c1" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597929    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597937    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597952    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.597958    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598034    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598043    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a6afbc-7263-4294-8cb8-62a42bc405c1" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598049    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eefbc24-a9ec-4baa-9e64-e7544568de8c" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598074    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2761ca3c-378c-4c78-b2c7-692ac3b01e6e" containerName="maas-api"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598083    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7346d10-91b5-4170-b050-43ff825e95a5" containerName="maas-api"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598091    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66728df-2ebd-48e8-8c32-607dfc52fcbb" containerName="payload-pre-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598097    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65cb9f09-c01d-4155-a2ae-fa72da8bfe6f" containerName="payload-processing"
Sep 06 06:29:23.598429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.598103    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc95d52-1024-48e8-934b-fa4c4d1734d7" containerName="manager"
Sep 06 06:29:23.600985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.600968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.604684 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604657    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 06:29:23.604800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604688    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 06:29:23.604800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604691    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xhzg2\""
Sep 06 06:29:23.604914 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.604971 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604957    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 06:29:23.605024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.604957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.605024 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.605006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcr5d\" (UniqueName: \"kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.611379 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.611353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:23.706201 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.706161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.706201 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.706201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.706456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.706225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcr5d\" (UniqueName: \"kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.708691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.708662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.708691 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.708682    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.714169 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.714144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcr5d\" (UniqueName: \"kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d\") pod \"maas-controller-55947c9784-nnw54\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") " pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:23.913011 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:23.912915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:24.185532 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:24.185509    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:24.187381 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:29:24.187353    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ccd16d_177a_4d63_b906_ee14fcb85d2b.slice/crio-e118c001e391f87edebdbe58ff5735444cad60bccd200df98fd9c5c848427b82 WatchSource:0}: Error finding container e118c001e391f87edebdbe58ff5735444cad60bccd200df98fd9c5c848427b82: Status 404 returned error can't find the container with id e118c001e391f87edebdbe58ff5735444cad60bccd200df98fd9c5c848427b82
Sep 06 06:29:24.188870 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:24.188847    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 06:29:25.086759 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:25.086670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-nnw54" event={"ID":"e8ccd16d-177a-4d63-b906-ee14fcb85d2b","Type":"ContainerStarted","Data":"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"}
Sep 06 06:29:25.086759 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:25.086725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-nnw54" event={"ID":"e8ccd16d-177a-4d63-b906-ee14fcb85d2b","Type":"ContainerStarted","Data":"e118c001e391f87edebdbe58ff5735444cad60bccd200df98fd9c5c848427b82"}
Sep 06 06:29:25.087228 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:25.086764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:25.103456 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:25.103399    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55947c9784-nnw54" podStartSLOduration=1.482643812 podStartE2EDuration="2.10338373s" totalImagesPullingTime="620.739918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:29:25.101670668 +0000 UTC m=+2118.805223404" watchObservedRunningTime="2026-09-06 06:29:25.10338373 +0000 UTC m=+2118.806936465"
Sep 06 06:29:36.098007 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:36.097974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:44.886751 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:44.886710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:29:45.326108 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.326054    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:29:45.421873 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.421838    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:45.422125 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.422051    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55947c9784-nnw54" podUID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" containerName="manager" containerID="cri-o://d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85" gracePeriod=30
Sep 06 06:29:45.676593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.676569    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:45.780266 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.780236    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert\") pod \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") "
Sep 06 06:29:45.780462 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.780273    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcr5d\" (UniqueName: \"kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d\") pod \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") "
Sep 06 06:29:45.780462 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.780321    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs\") pod \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\" (UID: \"e8ccd16d-177a-4d63-b906-ee14fcb85d2b\") "
Sep 06 06:29:45.782447 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.782412    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8ccd16d-177a-4d63-b906-ee14fcb85d2b" (UID: "e8ccd16d-177a-4d63-b906-ee14fcb85d2b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:29:45.782554 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.782449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e8ccd16d-177a-4d63-b906-ee14fcb85d2b" (UID: "e8ccd16d-177a-4d63-b906-ee14fcb85d2b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:29:45.782554 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.782463    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d" (OuterVolumeSpecName: "kube-api-access-jcr5d") pod "e8ccd16d-177a-4d63-b906-ee14fcb85d2b" (UID: "e8ccd16d-177a-4d63-b906-ee14fcb85d2b"). InnerVolumeSpecName "kube-api-access-jcr5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:29:45.881433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.881350    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-webhook-cert\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:29:45.881433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.881382    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcr5d\" (UniqueName: \"kubernetes.io/projected/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-kube-api-access-jcr5d\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:29:45.881433 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:45.881391    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8ccd16d-177a-4d63-b906-ee14fcb85d2b-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:29:46.192170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.192078    2567 generic.go:356] "Generic (PLEG): container finished" podID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" containerID="d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85" exitCode=0
Sep 06 06:29:46.192170 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.192122    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-nnw54"
Sep 06 06:29:46.192598 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.192166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-nnw54" event={"ID":"e8ccd16d-177a-4d63-b906-ee14fcb85d2b","Type":"ContainerDied","Data":"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"}
Sep 06 06:29:46.192598 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.192219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-nnw54" event={"ID":"e8ccd16d-177a-4d63-b906-ee14fcb85d2b","Type":"ContainerDied","Data":"e118c001e391f87edebdbe58ff5735444cad60bccd200df98fd9c5c848427b82"}
Sep 06 06:29:46.192598 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.192243    2567 scope.go:117] "RemoveContainer" containerID="d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"
Sep 06 06:29:46.201209 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.201180    2567 scope.go:117] "RemoveContainer" containerID="d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"
Sep 06 06:29:46.201438 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:29:46.201420    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85\": container with ID starting with d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85 not found: ID does not exist" containerID="d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"
Sep 06 06:29:46.201493 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.201448    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85"} err="failed to get container status \"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85\": rpc error: code = NotFound desc = could not find container \"d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85\": container with ID starting with d65dec3b32593de1ce8093e00d1703fd19c2cf7176bb1deb2a60b5d0114d7a85 not found: ID does not exist"
Sep 06 06:29:46.213269 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.213246    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:46.216732 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.216710    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55947c9784-nnw54"]
Sep 06 06:29:46.786917 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.786885    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" path="/var/lib/kubelet/pods/e8ccd16d-177a-4d63-b906-ee14fcb85d2b/volumes"
Sep 06 06:29:46.858485 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.858447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:29:46.974357 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.974313    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55947c9784-2bstg"]
Sep 06 06:29:46.974735 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.974715    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" containerName="manager"
Sep 06 06:29:46.974826 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.974738    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" containerName="manager"
Sep 06 06:29:46.974883 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.974830    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="721aa5f0-1d56-4a34-aef7-534353ae35b3" containerName="istio-proxy"
Sep 06 06:29:46.974883 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.974842    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ccd16d-177a-4d63-b906-ee14fcb85d2b" containerName="manager"
Sep 06 06:29:46.979125 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.979103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:46.982985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.982959    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 06:29:46.983128 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.982967    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 06:29:46.983211 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.983125    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 06:29:46.983211 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.983130    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xhzg2\""
Sep 06 06:29:46.985889 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:46.985867    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-2bstg"]
Sep 06 06:29:47.090423 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.090390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2v66\" (UniqueName: \"kubernetes.io/projected/312eca73-2c73-4584-8405-0b0b4b57439c-kube-api-access-b2v66\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.090580 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.090455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-metrics-certs\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.090580 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.090511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-webhook-cert\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.191118 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.191074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-metrics-certs\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.191291 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.191145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-webhook-cert\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.191291 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.191174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2v66\" (UniqueName: \"kubernetes.io/projected/312eca73-2c73-4584-8405-0b0b4b57439c-kube-api-access-b2v66\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.193648 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.193614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-webhook-cert\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.193985 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.193754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/312eca73-2c73-4584-8405-0b0b4b57439c-metrics-certs\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.198963 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.198936    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2v66\" (UniqueName: \"kubernetes.io/projected/312eca73-2c73-4584-8405-0b0b4b57439c-kube-api-access-b2v66\") pod \"maas-controller-55947c9784-2bstg\" (UID: \"312eca73-2c73-4584-8405-0b0b4b57439c\") " pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.291692 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.291663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:47.563329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:47.563294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55947c9784-2bstg"]
Sep 06 06:29:47.566506 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:29:47.566478    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312eca73_2c73_4584_8405_0b0b4b57439c.slice/crio-43820e7529ceacb6f8dd71ddec8ad066f09ea790d821c54045063f2c196cffd4 WatchSource:0}: Error finding container 43820e7529ceacb6f8dd71ddec8ad066f09ea790d821c54045063f2c196cffd4: Status 404 returned error can't find the container with id 43820e7529ceacb6f8dd71ddec8ad066f09ea790d821c54045063f2c196cffd4
Sep 06 06:29:48.204210 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:48.204118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-2bstg" event={"ID":"312eca73-2c73-4584-8405-0b0b4b57439c","Type":"ContainerStarted","Data":"489928d9f613304c9e29df7feeb1a5e2b80a754d8d036c17e74035a597642336"}
Sep 06 06:29:48.204210 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:48.204210    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:29:48.204656 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:48.204233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55947c9784-2bstg" event={"ID":"312eca73-2c73-4584-8405-0b0b4b57439c","Type":"ContainerStarted","Data":"43820e7529ceacb6f8dd71ddec8ad066f09ea790d821c54045063f2c196cffd4"}
Sep 06 06:29:48.221369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:48.221321    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55947c9784-2bstg" podStartSLOduration=1.911255393 podStartE2EDuration="2.221307058s" totalImagesPullingTime="310.051665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:29:48.220425042 +0000 UTC m=+2141.923977777" watchObservedRunningTime="2026-09-06 06:29:48.221307058 +0000 UTC m=+2141.924859794"
Sep 06 06:29:59.211663 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:29:59.211585    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55947c9784-2bstg"
Sep 06 06:30:54.013132 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:54.013086    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:30:54.013629 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:54.013297    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" podUID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" containerName="manager" containerID="cri-o://d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af" gracePeriod=10
Sep 06 06:30:55.068437 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.068414    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:30:55.171999 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.171916    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume\") pod \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") "
Sep 06 06:30:55.172160 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.172001    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g65lv\" (UniqueName: \"kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv\") pod \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\" (UID: \"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970\") "
Sep 06 06:30:55.172386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.172366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" (UID: "bb06f7e1-01ab-4fdd-a69c-c9f8ead58970"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:30:55.174009 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.173983    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv" (OuterVolumeSpecName: "kube-api-access-g65lv") pod "bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" (UID: "bb06f7e1-01ab-4fdd-a69c-c9f8ead58970"). InnerVolumeSpecName "kube-api-access-g65lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:30:55.272868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.272834    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-extensions-socket-volume\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:30:55.272868 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.272861    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g65lv\" (UniqueName: \"kubernetes.io/projected/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970-kube-api-access-g65lv\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:30:55.534500 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.534419    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" containerID="d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af" exitCode=0
Sep 06 06:30:55.534500 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.534461    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"
Sep 06 06:30:55.534500 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.534492    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" event={"ID":"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970","Type":"ContainerDied","Data":"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"}
Sep 06 06:30:55.534719 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.534533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86" event={"ID":"bb06f7e1-01ab-4fdd-a69c-c9f8ead58970","Type":"ContainerDied","Data":"527d47c4ad827fced4e4a59fa6f602e46f92015b9432305fdbc6e9ad4f5f5b7f"}
Sep 06 06:30:55.534719 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.534555    2567 scope.go:117] "RemoveContainer" containerID="d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"
Sep 06 06:30:55.544224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.544205    2567 scope.go:117] "RemoveContainer" containerID="d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"
Sep 06 06:30:55.544480 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:30:55.544463    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af\": container with ID starting with d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af not found: ID does not exist" containerID="d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"
Sep 06 06:30:55.544529 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.544488    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af"} err="failed to get container status \"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af\": rpc error: code = NotFound desc = could not find container \"d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af\": container with ID starting with d1747e7bd9e4f170ddc1f353e06e869e35dae65a3726bae161c452f8211d65af not found: ID does not exist"
Sep 06 06:30:55.556633 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.556606    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:30:55.560441 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:55.560418    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6lt86"]
Sep 06 06:30:56.786426 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:30:56.786394    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" path="/var/lib/kubelet/pods/bb06f7e1-01ab-4fdd-a69c-c9f8ead58970/volumes"
Sep 06 06:31:54.672029 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.671985    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"]
Sep 06 06:31:54.672475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.672327    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" containerName="manager"
Sep 06 06:31:54.672475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.672338    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" containerName="manager"
Sep 06 06:31:54.672475 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.672394    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb06f7e1-01ab-4fdd-a69c-c9f8ead58970" containerName="manager"
Sep 06 06:31:54.675209 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.675189    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.677903 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.677881    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 06:31:54.678101 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.678053    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 06:31:54.678212 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.678164    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m6mpq\""
Sep 06 06:31:54.690420 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.690396    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"]
Sep 06 06:31:54.777926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.777896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3577fd4c-a690-4c84-8182-4d74abcb1bba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.777926 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.777928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7zzj\" (UniqueName: \"kubernetes.io/projected/3577fd4c-a690-4c84-8182-4d74abcb1bba-kube-api-access-b7zzj\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.879368 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.879336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3577fd4c-a690-4c84-8182-4d74abcb1bba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.879562 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.879374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7zzj\" (UniqueName: \"kubernetes.io/projected/3577fd4c-a690-4c84-8182-4d74abcb1bba-kube-api-access-b7zzj\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.879716 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.879691    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3577fd4c-a690-4c84-8182-4d74abcb1bba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.888451 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.888429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7zzj\" (UniqueName: \"kubernetes.io/projected/3577fd4c-a690-4c84-8182-4d74abcb1bba-kube-api-access-b7zzj\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-htc4g\" (UID: \"3577fd4c-a690-4c84-8182-4d74abcb1bba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:54.984742 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:54.984663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:55.263492 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:55.263462    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"]
Sep 06 06:31:55.265227 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:31:55.265197    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3577fd4c_a690_4c84_8182_4d74abcb1bba.slice/crio-fd87eb85ffca5708b18b61efd28c55e64824d2efbdce79d411b7ebb83e965bd4 WatchSource:0}: Error finding container fd87eb85ffca5708b18b61efd28c55e64824d2efbdce79d411b7ebb83e965bd4: Status 404 returned error can't find the container with id fd87eb85ffca5708b18b61efd28c55e64824d2efbdce79d411b7ebb83e965bd4
Sep 06 06:31:55.820375 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:55.820336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g" event={"ID":"3577fd4c-a690-4c84-8182-4d74abcb1bba","Type":"ContainerStarted","Data":"d0a9fd4f35ffbe4ba4088c22f14d07cebafa1160250b4b8e97ec391c1c73ef98"}
Sep 06 06:31:55.820752 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:55.820392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g" event={"ID":"3577fd4c-a690-4c84-8182-4d74abcb1bba","Type":"ContainerStarted","Data":"fd87eb85ffca5708b18b61efd28c55e64824d2efbdce79d411b7ebb83e965bd4"}
Sep 06 06:31:55.820752 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:55.820415    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:31:55.841245 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:31:55.841193    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g" podStartSLOduration=1.8411786669999999 podStartE2EDuration="1.841178667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:31:55.839942834 +0000 UTC m=+2269.543495569" watchObservedRunningTime="2026-09-06 06:31:55.841178667 +0000 UTC m=+2269.544731402"
Sep 06 06:32:06.827865 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:06.827833    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-htc4g"
Sep 06 06:32:33.192707 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.192669    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:32:33.196292 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.196274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.199653 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.199628    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-555120e3-kserve-self-signed-certs\""
Sep 06 06:32:33.206357 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.206330    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:32:33.253124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.253090    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:32:33.284369 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284334    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.284518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.284518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwc55\" (UniqueName: \"kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.284518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.284518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.284658 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.284541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwc55\" (UniqueName: \"kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385539 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385819 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385884 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.385957 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.385926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.387760 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.387741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.387947 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.387930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.393055 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.393032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwc55\" (UniqueName: \"kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55\") pod \"e2e-same-id-a-555120e3-kserve-685f48d975-2mh58\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") " pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.507287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.507211    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:33.778722 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:33.778693    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:32:33.780790 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:32:33.780763    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf581d674_c54c_4f70_84dd_5b5e7e833993.slice/crio-2567ebd0d1e3d50171522d0df91c0c39c660629f5322f285413a2d7bdfb67327 WatchSource:0}: Error finding container 2567ebd0d1e3d50171522d0df91c0c39c660629f5322f285413a2d7bdfb67327: Status 404 returned error can't find the container with id 2567ebd0d1e3d50171522d0df91c0c39c660629f5322f285413a2d7bdfb67327
Sep 06 06:32:34.008333 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:34.008296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" event={"ID":"f581d674-c54c-4f70-84dd-5b5e7e833993","Type":"ContainerStarted","Data":"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"}
Sep 06 06:32:34.008333 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:34.008336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" event={"ID":"f581d674-c54c-4f70-84dd-5b5e7e833993","Type":"ContainerStarted","Data":"2567ebd0d1e3d50171522d0df91c0c39c660629f5322f285413a2d7bdfb67327"}
Sep 06 06:32:34.025290 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:34.025237    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" podStartSLOduration=1.025216787 podStartE2EDuration="1.025216787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:32:34.023893839 +0000 UTC m=+2307.727446586" watchObservedRunningTime="2026-09-06 06:32:34.025216787 +0000 UTC m=+2307.728769522"
Sep 06 06:32:43.508102 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:43.508052    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:43.508693 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:43.508125    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:43.520434 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:43.520411    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:44.068408 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:44.068376    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:32:45.983257 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:45.983220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:32:46.084173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:46.084144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:32:59.204365 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.204275    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:32:59.208224 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.208197    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.210992 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.210964    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-555120e3-kserve-self-signed-certs\""
Sep 06 06:32:59.219730 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.219700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:32:59.231332 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.231301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:32:59.314623 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.314823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.314823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.314823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.314823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.314823 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.314818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6465h\" (UniqueName: \"kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416076 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416105    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416287 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6465h\" (UniqueName: \"kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416602 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416602 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.416709 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.416644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.418424 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.418400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.418727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.418707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.423518 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.423492    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6465h\" (UniqueName: \"kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h\") pod \"e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") " pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.520473 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.520373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:32:59.795748 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:32:59.795660    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:32:59.798737 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:32:59.798707    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac2f3b9f_6aa4_4eb6_acd4_d4a2f83ca198.slice/crio-a7ffab075abc0c40f99c1d174195e94b3cb35638c4bc117a578c1113827999a1 WatchSource:0}: Error finding container a7ffab075abc0c40f99c1d174195e94b3cb35638c4bc117a578c1113827999a1: Status 404 returned error can't find the container with id a7ffab075abc0c40f99c1d174195e94b3cb35638c4bc117a578c1113827999a1
Sep 06 06:33:00.139910 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:00.139866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" event={"ID":"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198","Type":"ContainerStarted","Data":"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"}
Sep 06 06:33:00.140130 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:00.139925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" event={"ID":"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198","Type":"ContainerStarted","Data":"a7ffab075abc0c40f99c1d174195e94b3cb35638c4bc117a578c1113827999a1"}
Sep 06 06:33:00.157792 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:00.157725    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" podStartSLOduration=1.157707508 podStartE2EDuration="1.157707508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:33:00.156142245 +0000 UTC m=+2333.859694984" watchObservedRunningTime="2026-09-06 06:33:00.157707508 +0000 UTC m=+2333.861260241"
Sep 06 06:33:05.306603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:05.306548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:33:09.521153 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:09.521113    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:09.521644 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:09.521171    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:09.536331 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:09.536296    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:10.200552 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:10.200516    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:27.759429 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:27.759394    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:33:27.778813 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:27.778782    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:33:28.511965 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:28.511929    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:33:28.532033 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:28.532003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:33:42.781627 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:42.781544    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" podUID="f581d674-c54c-4f70-84dd-5b5e7e833993" containerName="main" containerID="cri-o://60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044" gracePeriod=45
Sep 06 06:33:43.026933 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.026903    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:33:43.104425 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104397    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104479    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104494    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104515    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104546    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwc55\" (UniqueName: \"kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104586    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache\") pod \"f581d674-c54c-4f70-84dd-5b5e7e833993\" (UID: \"f581d674-c54c-4f70-84dd-5b5e7e833993\") "
Sep 06 06:33:43.104938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.104938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104804    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home" (OuterVolumeSpecName: "home") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.104938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104873    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache" (OuterVolumeSpecName: "model-cache") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.104938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.104878    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.106850 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.106824    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm" (OuterVolumeSpecName: "dshm") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.106948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.106828    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:33:43.106948 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.106849    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55" (OuterVolumeSpecName: "kube-api-access-lwc55") pod "f581d674-c54c-4f70-84dd-5b5e7e833993" (UID: "f581d674-c54c-4f70-84dd-5b5e7e833993"). InnerVolumeSpecName "kube-api-access-lwc55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:33:43.205833 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.205798    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.205833 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.205827    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.205833 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.205836    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f581d674-c54c-4f70-84dd-5b5e7e833993-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.206050 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.205847    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwc55\" (UniqueName: \"kubernetes.io/projected/f581d674-c54c-4f70-84dd-5b5e7e833993-kube-api-access-lwc55\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.206050 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.205857    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f581d674-c54c-4f70-84dd-5b5e7e833993-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:43.360754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.360664    2567 generic.go:356] "Generic (PLEG): container finished" podID="f581d674-c54c-4f70-84dd-5b5e7e833993" containerID="60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044" exitCode=0
Sep 06 06:33:43.360754 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.360709    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"
Sep 06 06:33:43.360944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.360748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" event={"ID":"f581d674-c54c-4f70-84dd-5b5e7e833993","Type":"ContainerDied","Data":"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"}
Sep 06 06:33:43.360944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.360792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58" event={"ID":"f581d674-c54c-4f70-84dd-5b5e7e833993","Type":"ContainerDied","Data":"2567ebd0d1e3d50171522d0df91c0c39c660629f5322f285413a2d7bdfb67327"}
Sep 06 06:33:43.360944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.360808    2567 scope.go:117] "RemoveContainer" containerID="60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"
Sep 06 06:33:43.369858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.369841    2567 scope.go:117] "RemoveContainer" containerID="60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"
Sep 06 06:33:43.370167 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:33:43.370145    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044\": container with ID starting with 60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044 not found: ID does not exist" containerID="60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"
Sep 06 06:33:43.370227 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.370180    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044"} err="failed to get container status \"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044\": rpc error: code = NotFound desc = could not find container \"60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044\": container with ID starting with 60e0593454f7485f4f7a146b65af1cffeab68150f32c170f0522a7368b686044 not found: ID does not exist"
Sep 06 06:33:43.381280 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.381257    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:33:43.384486 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.384463    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-555120e3-kserve-685f48d975-2mh58"]
Sep 06 06:33:43.538398 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.538356    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" podUID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" containerName="main" containerID="cri-o://0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d" gracePeriod=45
Sep 06 06:33:43.774800 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.774773    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:43.911128 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911024    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911128 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911158    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911204    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6465h\" (UniqueName: \"kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911233    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir\") pod \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\" (UID: \"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198\") "
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911465    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home" (OuterVolumeSpecName: "home") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911480    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.911614 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.911529    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.913276 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.913247    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm" (OuterVolumeSpecName: "dshm") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 06:33:43.913358 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.913329    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h" (OuterVolumeSpecName: "kube-api-access-6465h") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "kube-api-access-6465h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:33:43.913397 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:43.913359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" (UID: "ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:33:44.012185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012142    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-home\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.012185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012172    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-model-cache\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.012185 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012187    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6465h\" (UniqueName: \"kubernetes.io/projected/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-kube-api-access-6465h\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.012417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012200    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tmp-dir\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.012417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012208    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-dshm\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.012417 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.012216    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198-tls-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:44.366801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.366765    2567 generic.go:356] "Generic (PLEG): container finished" podID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" containerID="0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d" exitCode=0
Sep 06 06:33:44.366991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.366806    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"
Sep 06 06:33:44.366991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.366846    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" event={"ID":"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198","Type":"ContainerDied","Data":"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"}
Sep 06 06:33:44.366991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.366887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66" event={"ID":"ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198","Type":"ContainerDied","Data":"a7ffab075abc0c40f99c1d174195e94b3cb35638c4bc117a578c1113827999a1"}
Sep 06 06:33:44.366991 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.366910    2567 scope.go:117] "RemoveContainer" containerID="0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"
Sep 06 06:33:44.378746 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.378718    2567 scope.go:117] "RemoveContainer" containerID="0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"
Sep 06 06:33:44.379114 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:33:44.379091    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d\": container with ID starting with 0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d not found: ID does not exist" containerID="0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"
Sep 06 06:33:44.379206 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.379123    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d"} err="failed to get container status \"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d\": rpc error: code = NotFound desc = could not find container \"0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d\": container with ID starting with 0d6ff6acf616cd17fa3b5bd639f02f5425de76ca6e99490be3d25879a8d7756d not found: ID does not exist"
Sep 06 06:33:44.388905 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.388877    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:33:44.392555 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.392529    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-555120e3-kserve-69b54d84cf-c5h66"]
Sep 06 06:33:44.787212 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.787131    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" path="/var/lib/kubelet/pods/ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198/volumes"
Sep 06 06:33:44.787546 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:44.787533    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f581d674-c54c-4f70-84dd-5b5e7e833993" path="/var/lib/kubelet/pods/f581d674-c54c-4f70-84dd-5b5e7e833993/volumes"
Sep 06 06:33:45.321285 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321248    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"]
Sep 06 06:33:45.321771 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321750    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" containerName="main"
Sep 06 06:33:45.321843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321774    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" containerName="main"
Sep 06 06:33:45.321843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321804    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f581d674-c54c-4f70-84dd-5b5e7e833993" containerName="main"
Sep 06 06:33:45.321843 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321813    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f581d674-c54c-4f70-84dd-5b5e7e833993" containerName="main"
Sep 06 06:33:45.321981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321894    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac2f3b9f-6aa4-4eb6-acd4-d4a2f83ca198" containerName="main"
Sep 06 06:33:45.321981 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.321912    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f581d674-c54c-4f70-84dd-5b5e7e833993" containerName="main"
Sep 06 06:33:45.326607 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.326583    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.332394 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.332005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"]
Sep 06 06:33:45.427314 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.427274    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwqf\" (UniqueName: \"kubernetes.io/projected/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-kube-api-access-nhwqf\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.427510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.427367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-metrics-certs\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.427510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.427419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-maas-api-tls\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.427616 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.427502    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-openshift-service-ca\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.528372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.528331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwqf\" (UniqueName: \"kubernetes.io/projected/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-kube-api-access-nhwqf\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.528603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.528404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-metrics-certs\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.528603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.528446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-maas-api-tls\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.528603 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.528479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-openshift-service-ca\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.529322 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.529287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-openshift-service-ca\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.531197 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.531173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-metrics-certs\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.531415 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.531309    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-maas-api-tls\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.536351 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.536325    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwqf\" (UniqueName: \"kubernetes.io/projected/b440fb23-0a8a-4f7d-b43a-f0f43b4135f4-kube-api-access-nhwqf\") pod \"maas-api-7c946f6b4d-g9lcx\" (UID: \"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4\") " pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.644649 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.644555    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:45.919880 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:45.919853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"]
Sep 06 06:33:45.921456 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:33:45.921429    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb440fb23_0a8a_4f7d_b43a_f0f43b4135f4.slice/crio-24b40fce436e6ac95a680708d7b14aa3267814a40f63cd56ad5c9714ec8f2333 WatchSource:0}: Error finding container 24b40fce436e6ac95a680708d7b14aa3267814a40f63cd56ad5c9714ec8f2333: Status 404 returned error can't find the container with id 24b40fce436e6ac95a680708d7b14aa3267814a40f63cd56ad5c9714ec8f2333
Sep 06 06:33:46.379699 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:46.379663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx" event={"ID":"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4","Type":"ContainerStarted","Data":"24b40fce436e6ac95a680708d7b14aa3267814a40f63cd56ad5c9714ec8f2333"}
Sep 06 06:33:46.958951 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:46.958911    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:33:47.029670 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:47.029628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:33:47.386317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:47.386277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx" event={"ID":"b440fb23-0a8a-4f7d-b43a-f0f43b4135f4","Type":"ContainerStarted","Data":"806308c698b2542d4ad0296a7638569cb1f3bb2a90ef90ced844dcbab007f732"}
Sep 06 06:33:47.386748 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:47.386350    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:47.405025 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:47.404969    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx" podStartSLOduration=1.983448096 podStartE2EDuration="2.404949815s" totalImagesPullingTime="421.501719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 06:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:33:47.402774214 +0000 UTC m=+2381.106326950" watchObservedRunningTime="2026-09-06 06:33:47.404949815 +0000 UTC m=+2381.108502611"
Sep 06 06:33:53.397076 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:53.397031    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7c946f6b4d-g9lcx"
Sep 06 06:33:53.441392 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:53.441359    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:33:53.441712 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:53.441673    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" podUID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" containerName="maas-api" containerID="cri-o://7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422" gracePeriod=30
Sep 06 06:33:54.683442 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.683415    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:33:54.710300 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.710269    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca\") pod \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") "
Sep 06 06:33:54.710428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.710312    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm5gl\" (UniqueName: \"kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl\") pod \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") "
Sep 06 06:33:54.710428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.710366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") pod \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") "
Sep 06 06:33:54.710428 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.710396    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs\") pod \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\" (UID: \"641aaab5-0cd4-4016-bd88-fb349a6bfed6\") "
Sep 06 06:33:54.710718 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.710681    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "641aaab5-0cd4-4016-bd88-fb349a6bfed6" (UID: "641aaab5-0cd4-4016-bd88-fb349a6bfed6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 06:33:54.712559 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.712533    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "641aaab5-0cd4-4016-bd88-fb349a6bfed6" (UID: "641aaab5-0cd4-4016-bd88-fb349a6bfed6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:33:54.712678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.712597    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "641aaab5-0cd4-4016-bd88-fb349a6bfed6" (UID: "641aaab5-0cd4-4016-bd88-fb349a6bfed6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 06:33:54.712678 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.712644    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl" (OuterVolumeSpecName: "kube-api-access-xm5gl") pod "641aaab5-0cd4-4016-bd88-fb349a6bfed6" (UID: "641aaab5-0cd4-4016-bd88-fb349a6bfed6"). InnerVolumeSpecName "kube-api-access-xm5gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 06:33:54.811695 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.811668    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-maas-api-tls\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:54.811695 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.811691    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/641aaab5-0cd4-4016-bd88-fb349a6bfed6-metrics-certs\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:54.811837 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.811702    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641aaab5-0cd4-4016-bd88-fb349a6bfed6-openshift-service-ca\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:54.811837 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:54.811710    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm5gl\" (UniqueName: \"kubernetes.io/projected/641aaab5-0cd4-4016-bd88-fb349a6bfed6-kube-api-access-xm5gl\") on node \"ip-10-0-129-33.ec2.internal\" DevicePath \"\""
Sep 06 06:33:55.428173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.428137    2567 generic.go:356] "Generic (PLEG): container finished" podID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" containerID="7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422" exitCode=0
Sep 06 06:33:55.428372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.428184    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"
Sep 06 06:33:55.428372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.428219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" event={"ID":"641aaab5-0cd4-4016-bd88-fb349a6bfed6","Type":"ContainerDied","Data":"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"}
Sep 06 06:33:55.428372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.428281    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76f875657b-fd6jd" event={"ID":"641aaab5-0cd4-4016-bd88-fb349a6bfed6","Type":"ContainerDied","Data":"0c74338e7340f50e5c634637e8da6d4d4a8e8d4bc876afe701bdc4b2555dacb4"}
Sep 06 06:33:55.428372 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.428306    2567 scope.go:117] "RemoveContainer" containerID="7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"
Sep 06 06:33:55.436912 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.436894    2567 scope.go:117] "RemoveContainer" containerID="7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"
Sep 06 06:33:55.437196 ip-10-0-129-33 kubenswrapper[2567]: E0906 06:33:55.437176    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422\": container with ID starting with 7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422 not found: ID does not exist" containerID="7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"
Sep 06 06:33:55.437242 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.437205    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422"} err="failed to get container status \"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422\": rpc error: code = NotFound desc = could not find container \"7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422\": container with ID starting with 7bae48aea8622b57f517288517196e4ff7f0f6bfacff50fc34a33637693b5422 not found: ID does not exist"
Sep 06 06:33:55.457719 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.457691    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:33:55.459686 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:55.459666    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-76f875657b-fd6jd"]
Sep 06 06:33:56.786920 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:33:56.786886    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" path="/var/lib/kubelet/pods/641aaab5-0cd4-4016-bd88-fb349a6bfed6/volumes"
Sep 06 06:34:20.424804 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:20.424770    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wpf4d_94ff23f0-bb45-48aa-a7bd-4132cbfd0e81/manager/0.log"
Sep 06 06:34:20.671386 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:20.671358    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55947c9784-2bstg_312eca73-2c73-4584-8405-0b0b4b57439c/manager/0.log"
Sep 06 06:34:20.786503 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:20.786434    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p9tsq_1ebbd126-638a-46b5-9cbf-44ba12db945c/server/0.log"
Sep 06 06:34:21.140305 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:21.140278    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b897cd959-l9k4j_56df9ec4-77bc-491e-b774-0da95d7c4a5a/copy-manifests/0.log"
Sep 06 06:34:21.151855 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:21.151816    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b897cd959-l9k4j_56df9ec4-77bc-491e-b774-0da95d7c4a5a/manager/0.log"
Sep 06 06:34:21.698778 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:21.698750    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7c946f6b4d-g9lcx_b440fb23-0a8a-4f7d-b43a-f0f43b4135f4/maas-api/0.log"
Sep 06 06:34:22.027124 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:22.027020    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-59tkk_d0fb8daa-3e1b-4117-9121-7430ea2c9f9e/postgres/0.log"
Sep 06 06:34:23.220624 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:23.220594    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-htc4g_3577fd4c-a690-4c84-8182-4d74abcb1bba/manager/0.log"
Sep 06 06:34:23.899418 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:23.899390    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d2jb4_94ccf2fc-7a70-46d8-b1e9-606aebe798d9/discovery/0.log"
Sep 06 06:34:24.008777 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:24.008742    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696f56895b-6rlkm_addb6816-ed73-4db1-a01e-2c4b72b0b488/kube-auth-proxy/0.log"
Sep 06 06:34:24.339330 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:24.339292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8rpm6_9c49b2e6-4c6a-4cb7-911e-ecbb553b8121/payload-pre-processing/0.log"
Sep 06 06:34:24.903593 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:24.903561    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-p6xvb_54ba2c57-489b-4d7f-9aa1-82cf2dcc7cec/main/0.log"
Sep 06 06:34:25.012577 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:25.012548    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-k7xst_d8091f7c-9b4d-4ea9-b574-74561e0d2df7/main/0.log"
Sep 06 06:34:31.084502 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:31.084426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:34:32.315419 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:32.315385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log" limit=1024
Sep 06 06:34:34.045769 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:34.045747    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wsczd_8eeb7275-b0a1-4dca-bc64-979136db13eb/global-pull-secret-syncer/0.log"
Sep 06 06:34:34.092309 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:34.092280    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8kdpv_b23f16e8-d43a-4b61-bb80-b7e0c39d5c25/konnectivity-agent/0.log"
Sep 06 06:34:34.170472 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:34.170445    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-33.ec2.internal_00d162c63e419062586c3987b0b951dc/haproxy/0.log"
Sep 06 06:34:39.568790 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:39.568757    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-htc4g_3577fd4c-a690-4c84-8182-4d74abcb1bba/manager/0.log"
Sep 06 06:34:41.295588 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:41.295556    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vpss_f008550a-4455-48c0-b995-228efae0b97a/node-exporter/0.log"
Sep 06 06:34:41.317014 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:41.316984    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vpss_f008550a-4455-48c0-b995-228efae0b97a/kube-rbac-proxy/0.log"
Sep 06 06:34:41.338397 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:41.338371    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vpss_f008550a-4455-48c0-b995-228efae0b97a/init-textfile/0.log"
Sep 06 06:34:44.685863 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.685829    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"]
Sep 06 06:34:44.686249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.686186    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" containerName="maas-api"
Sep 06 06:34:44.686249 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.686201    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" containerName="maas-api"
Sep 06 06:34:44.686328 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.686285    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="641aaab5-0cd4-4016-bd88-fb349a6bfed6" containerName="maas-api"
Sep 06 06:34:44.689261 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.689241    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.691954 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.691924    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zzhm6\"/\"kube-root-ca.crt\""
Sep 06 06:34:44.693157 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.693117    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zzhm6\"/\"openshift-service-ca.crt\""
Sep 06 06:34:44.693157 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.693137    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zzhm6\"/\"default-dockercfg-gv2n8\""
Sep 06 06:34:44.698703 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.698680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"]
Sep 06 06:34:44.745651 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.745622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-sys\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.745656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg66d\" (UniqueName: \"kubernetes.io/projected/256f7651-f5c9-426d-b17e-2882851c7374-kube-api-access-vg66d\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.745689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-proc\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.745711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-lib-modules\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.745801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.745726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-podres\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.846944 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.846906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-sys\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.846946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg66d\" (UniqueName: \"kubernetes.io/projected/256f7651-f5c9-426d-b17e-2882851c7374-kube-api-access-vg66d\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-proc\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-sys\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-proc\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-lib-modules\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-podres\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847317 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-lib-modules\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.847479 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.847434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/256f7651-f5c9-426d-b17e-2882851c7374-podres\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:44.855344 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:44.855328    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg66d\" (UniqueName: \"kubernetes.io/projected/256f7651-f5c9-426d-b17e-2882851c7374-kube-api-access-vg66d\") pod \"perf-node-gather-daemonset-96zqc\" (UID: \"256f7651-f5c9-426d-b17e-2882851c7374\") " pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:45.000220 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.000134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:45.272631 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.272596    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"]
Sep 06 06:34:45.274135 ip-10-0-129-33 kubenswrapper[2567]: W0906 06:34:45.274106    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod256f7651_f5c9_426d_b17e_2882851c7374.slice/crio-750e2d2855e37e3322dfaffc60db39fc3e69d58e5b25e421f8cf34f545340d00 WatchSource:0}: Error finding container 750e2d2855e37e3322dfaffc60db39fc3e69d58e5b25e421f8cf34f545340d00: Status 404 returned error can't find the container with id 750e2d2855e37e3322dfaffc60db39fc3e69d58e5b25e421f8cf34f545340d00
Sep 06 06:34:45.684729 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.684691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc" event={"ID":"256f7651-f5c9-426d-b17e-2882851c7374","Type":"ContainerStarted","Data":"9b9d1288677e2d54c48958a6c116e9d1219c49aeb70bc82ebea3c9a11fd32e00"}
Sep 06 06:34:45.684938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.684742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc" event={"ID":"256f7651-f5c9-426d-b17e-2882851c7374","Type":"ContainerStarted","Data":"750e2d2855e37e3322dfaffc60db39fc3e69d58e5b25e421f8cf34f545340d00"}
Sep 06 06:34:45.684938 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.684821    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:45.700113 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:45.700037    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc" podStartSLOduration=1.700022391 podStartE2EDuration="1.700022391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 06:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 06:34:45.698716156 +0000 UTC m=+2439.402268903" watchObservedRunningTime="2026-09-06 06:34:45.700022391 +0000 UTC m=+2439.403575124"
Sep 06 06:34:46.543263 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:46.543231    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlcmz_99895669-efaf-48a4-a29d-6ab0667aa167/dns/0.log"
Sep 06 06:34:46.565052 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:46.565030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlcmz_99895669-efaf-48a4-a29d-6ab0667aa167/kube-rbac-proxy/0.log"
Sep 06 06:34:46.662611 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:46.662583    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sq2fh_0c060475-db7c-41ec-ac4b-5840c313ef83/dns-node-resolver/0.log"
Sep 06 06:34:47.225886 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:47.225852    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5c9d4bb797-nqfwm_a08c080b-5ffe-4998-873e-ddef2b9e4607/registry/0.log"
Sep 06 06:34:47.293077 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:47.293019    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdf8l_d8ee0da7-abb2-4bcd-b504-6487a14049d6/node-ca/0.log"
Sep 06 06:34:48.467745 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:48.467713    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d2jb4_94ccf2fc-7a70-46d8-b1e9-606aebe798d9/discovery/0.log"
Sep 06 06:34:48.488693 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:48.488664    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696f56895b-6rlkm_addb6816-ed73-4db1-a01e-2c4b72b0b488/kube-auth-proxy/0.log"
Sep 06 06:34:48.588727 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:48.588697    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8rpm6_9c49b2e6-4c6a-4cb7-911e-ecbb553b8121/payload-pre-processing/0.log"
Sep 06 06:34:49.502697 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:49.502666    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b96bw_1df33f75-f9ac-4088-a708-5e570b02dea5/serve-healthcheck-canary/0.log"
Sep 06 06:34:50.032858 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:50.032827    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-26knw_7de92d74-d8f0-4ff5-881c-38fe64d20071/kube-rbac-proxy/0.log"
Sep 06 06:34:50.056457 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:50.056428    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-26knw_7de92d74-d8f0-4ff5-881c-38fe64d20071/exporter/0.log"
Sep 06 06:34:50.078315 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:50.078293    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-26knw_7de92d74-d8f0-4ff5-881c-38fe64d20071/extractor/0.log"
Sep 06 06:34:51.699040 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:51.699012    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zzhm6/perf-node-gather-daemonset-96zqc"
Sep 06 06:34:52.307527 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:52.307500    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wpf4d_94ff23f0-bb45-48aa-a7bd-4132cbfd0e81/manager/0.log"
Sep 06 06:34:52.592081 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:52.591991    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55947c9784-2bstg_312eca73-2c73-4584-8405-0b0b4b57439c/manager/0.log"
Sep 06 06:34:52.613683 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:52.613661    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p9tsq_1ebbd126-638a-46b5-9cbf-44ba12db945c/server/0.log"
Sep 06 06:34:52.842641 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:52.842557    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b897cd959-l9k4j_56df9ec4-77bc-491e-b774-0da95d7c4a5a/manager/0.log"
Sep 06 06:34:52.859822 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:52.859802    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b897cd959-l9k4j_56df9ec4-77bc-491e-b774-0da95d7c4a5a/copy-manifests/0.log"
Sep 06 06:34:55.492446 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:55.492411    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d2jb4_94ccf2fc-7a70-46d8-b1e9-606aebe798d9/discovery/0.log"
Sep 06 06:34:55.502329 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:55.502306    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696f56895b-6rlkm_addb6816-ed73-4db1-a01e-2c4b72b0b488/kube-auth-proxy/0.log"
Sep 06 06:34:55.569173 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:55.569136    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8rpm6_9c49b2e6-4c6a-4cb7-911e-ecbb553b8121/payload-pre-processing/0.log"
Sep 06 06:34:57.469790 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:57.469755    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d2jb4_94ccf2fc-7a70-46d8-b1e9-606aebe798d9/discovery/0.log"
Sep 06 06:34:57.484511 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:57.484488    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696f56895b-6rlkm_addb6816-ed73-4db1-a01e-2c4b72b0b488/kube-auth-proxy/0.log"
Sep 06 06:34:57.568182 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:34:57.568153    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8rpm6_9c49b2e6-4c6a-4cb7-911e-ecbb553b8121/payload-pre-processing/0.log"
Sep 06 06:35:01.378809 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.378776    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/kube-multus-additional-cni-plugins/0.log"
Sep 06 06:35:01.399801 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.399774    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/egress-router-binary-copy/0.log"
Sep 06 06:35:01.419967 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.419940    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/cni-plugins/0.log"
Sep 06 06:35:01.440510 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.440484    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/bond-cni-plugin/0.log"
Sep 06 06:35:01.461134 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.461108    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/routeoverride-cni/0.log"
Sep 06 06:35:01.481357 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.481337    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/whereabouts-cni-bincopy/0.log"
Sep 06 06:35:01.501100 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.501075    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4ktq_fa8e1de9-0276-49d2-ad3c-4c4e79826925/whereabouts-cni/0.log"
Sep 06 06:35:01.540954 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.540924    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cmg6t_98c6cf0e-08b2-4db0-9e50-6ebfec68eff3/kube-multus/0.log"
Sep 06 06:35:01.697072 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.696985    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pvwwb_4f1efb45-a0eb-4de5-a2d1-1557a281bab6/network-metrics-daemon/0.log"
Sep 06 06:35:01.715830 ip-10-0-129-33 kubenswrapper[2567]: I0906 06:35:01.715803    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pvwwb_4f1efb45-a0eb-4de5-a2d1-1557a281bab6/kube-rbac-proxy/0.log"